IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2402.13604.html
   My bibliography  Save this paper

Breaking the HISCO Barrier: Automatic Occupational Standardization with OccCANINE

Author

Listed:
  • Christian M{o}ller Dahl
  • Torben Johansen
  • Christian Vedel

Abstract

This paper introduces a new tool, OccCANINE, to automatically transform occupational descriptions into the HISCO classification system. The manual work involved in processing and classifying occupational descriptions is error-prone, tedious, and time-consuming. We finetune a preexisting language model (CANINE) to do this automatically, thereby performing in seconds and minutes what previously took days and weeks. The model is trained on 14 million pairs of occupational descriptions and HISCO codes in 13 different languages contributed by 22 different sources. Our approach is shown to have accuracy, recall, and precision above 90 percent. Our tool breaks the metaphorical HISCO barrier and makes this data readily available for analysis of occupational structures with broad applicability in economics, economic history, and various related disciplines.

Suggested Citation

  • Christian M{o}ller Dahl & Torben Johansen & Christian Vedel, 2024. "Breaking the HISCO Barrier: Automatic Occupational Standardization with OccCANINE," Papers 2402.13604, arXiv.org, revised Apr 2024.
    • Handle: RePEc:arx:papers:2402.13604
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2402.13604
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Allen,Robert C., 2009. "The British Industrial Revolution in Global Perspective," Cambridge Books, Cambridge University Press, number 9780521868273.
    2. Berger, Thor, 2019. "Railroads and Rural Industrialization: evidence from a Historical Policy Experiment," Explorations in Economic History, Elsevier, vol. 74(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jeanet Sinding Bentzen & Nina Boberg-Fazlić & Paul Sharp & Christian Volmar Skovsgaard & Christian Vedel, 2024. "Assimilate for God: The Impact of Religious Divisions on Danish American Communities," Working Papers 0253, European Historical Economics Society (EHES).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. DUDLEY, Leonard & RAUH, Christopher, 2018. "Innovation growth clusters: Lessons from the industrial revolution," Cahiers de recherche 2018-14, Universite de Montreal, Departement de sciences economiques.
    2. Schreiner, Lena & Madlener, Reinhard, 2022. "Investing in power grid infrastructure as a flexibility option: A DSGE assessment for Germany," Energy Economics, Elsevier, vol. 107(C).
    3. Thomas Baudin & Robert Stelter, 2022. "The rural exodus and the rise of Europe," Journal of Economic Growth, Springer, vol. 27(3), pages 365-414, September.
    4. Zamparelli, Luca, 2022. "On Labor Productivity Growth and the Wage Share with Endogenous Size and Direction of Technical Change," MPRA Paper 112684, University Library of Munich, Germany.
    5. Pamfili Antipa & Vincent Bignon, 2018. "Whither Economic History? Between Narratives and Quantification," Revue de l'OFCE, Presses de Sciences-Po, vol. 0(3), pages 17-36.
    6. Ferreira, Pedro Cavalcanti & Pessôa, Samuel & dos Santos, Marcelo Rodrigues, 2016. "Globalization And The Industrial Revolution," Macroeconomic Dynamics, Cambridge University Press, vol. 20(3), pages 643-666, April.
    7. Huning, Thilo R. & Wahl, Fabian, 2017. "Lord of the lemons: Origin and dynamics of state capacity," Hohenheim Discussion Papers in Business, Economics and Social Sciences 22-2017, University of Hohenheim, Faculty of Business, Economics and Social Sciences.
    8. Klas Rönnbäck, 2014. "Slave ownership and fossil fuel usage: a commentary," Climatic Change, Springer, vol. 122(1), pages 1-9, January.
    9. O'Brien, Patrick, 2018. "Cosmographies for the discovery, development and diffusion of useful and reliable knowledge in pre-industrial Europe and Late imperial China: a survey and speculation," Economic History Working Papers 90534, London School of Economics and Political Science, Department of Economic History.
    10. Daron Acemoglu & James A. Robinson, 2015. "The Rise and Decline of General Laws of Capitalism," Journal of Economic Perspectives, American Economic Association, vol. 29(1), pages 3-28, Winter.
    11. Dakpogan, Arnaud & Smit, Eon, 2018. "The effect of electricity losses on GDP in Benin," MPRA Paper 89545, University Library of Munich, Germany.
    12. Allen, Robert C., 2012. "Backward into the future: The shift to coal and implications for the next energy transition," Energy Policy, Elsevier, vol. 50(C), pages 17-23.
    13. Boško Mijatović & Branko Milanović, 2021. "The real urban wage in an agricultural economy without landless farmers: Serbia, 1862–1910," Economic History Review, Economic History Society, vol. 74(2), pages 424-448, May.
    14. Ewout Frankema & Marlous van Waijenburg, 2011. "African Real Wages in Asian Perspective, 1880-1940," Working Papers 0002, Utrecht University, Centre for Global Economic History.
    15. Palma, Nuno, 2018. "Money and modernization in early modern England," Financial History Review, Cambridge University Press, vol. 25(3), pages 231-261, December.
    16. Stephen Broadberry & Bruce Campbell & Alexander Klein & Mark Overton, 2010. "British economic growth, 1300-1850: some preliminary estimates," Working Papers 10009, Economic History Society.
    17. Cédric Chambru & Paul Maneuvrier‐Hervieu, 2023. "The evolution of wages in early modern Normandy (1600–1850)," Economic History Review, Economic History Society, vol. 76(3), pages 917-940, August.
    18. Vincent Geloso & Peter Lindert, 2020. "Relative costs of living, for richer and poorer, 1688–1914," Cliometrica, Springer;Cliometric Society (Association Francaise de Cliométrie), vol. 14(3), pages 417-442, September.
    19. Edwyna Harris & Sumner La Croix, 2019. "Prices, Wages, and Welfare in Early Colonial South Australia, 1836-1850," Monash Economics Working Papers 07-19, Monash University, Department of Economics.
    20. Broadberry, Stephen & Ghosal, Sayantan & Proto, Eugenio, 2017. "Anonymity, efficiency wages and technological progress," Journal of Development Economics, Elsevier, vol. 127(C), pages 379-394.

    More about this item

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • J1 - Labor and Demographic Economics - - Demographic Economics
    • N01 - Economic History - - General - - - Development of the Discipline: Historiographical; Sources and Methods
    • N3 - Economic History - - Labor and Consumers, Demography, Education, Health, Welfare, Income, Wealth, Religion, and Philanthropy
    • N6 - Economic History - - Manufacturing and Construction
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2402.13604. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.