IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/130163.html

How economics classifies itself: text-based JEL codes and their consistency

Author

Listed:
  • Garau, Alessio

Abstract

Can a language model improve how economists classify their own papers? Only 15% of four million IDEAS/RePEc records carry usable JEL codes, and similar papers often receive different ones. I use a large language model (LLM) to solve this problem and assign three-digit codes from titles and abstracts, evaluating it on 69,503 coded articles published from 1991 to 2023 in the top 100 economics journals. Two tests do not assume that author codes provide the correct classification. Across semantic neighbors identified by a separate embedding model, model codes are 1.8 times as consistent as author codes. Holding codes per paper fixed, a blind check finds that 83% of model codes fit official American Economic Association (AEA) guidelines, compared with 67% of author codes. The classifier expands coverage, and the evaluation framework applies whenever human labels are incomplete or noisy.

Suggested Citation

  • Garau, Alessio, 2026. "How economics classifies itself: text-based JEL codes and their consistency," MPRA Paper 130163, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:130163
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/130163/1/MPRA_paper_130163.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. David Card & Stefano DellaVigna & Patricia Funk & Nagore Iriberri, 2022. "Gender Differences in Peer Recognition by Economists," Econometrica, Econometric Society, vol. 90(5), pages 1937-1971, September.
    2. Josh Angrist & Pierre Azoulay & Glenn Ellison & Ryan Hill & Susan Feng Lu, 2020. "Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship," Journal of Economic Literature, American Economic Association, vol. 58(1), pages 3-52, March.
    3. Beatrice Cherrier, 2017. "Classifying Economics: A History of the JEL Codes," Journal of Economic Literature, American Economic Association, vol. 55(2), pages 545-579, June.
    4. Erin Hengel, 2022. "Publishing While Female: are Women Held to Higher Standards? Evidence from Peer Review," The Economic Journal, Royal Economic Society, vol. 132(648), pages 2951-2991.
    5. Wagstaff, Adam & Culyer, Anthony J., 2012. "Four decades of health economics through a bibliometric lens," Journal of Health Economics, Elsevier, vol. 31(2), pages 406-439.
    6. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    7. Michael A. Kelly & Stephen Bruestle, 2011. "Trend Of Subjects Published In Economics Journals 1969–2007," Economic Inquiry, Western Economic Association International, vol. 49(3), pages 658-673, July.
    8. Marlène Koffi, 2025. "Innovative Ideas and Gender (In)equality," American Economic Review, American Economic Association, vol. 115(7), pages 2207-2236, July.
    9. David Card & Stefano DellaVigna, 2020. "What Do Editors Maximize? Evidence from Four Economics Journals," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 195-217, March.
    10. Lea†Rachel Kosnik, 2018. "A Survey Of Jel Codes: What Do They Mean And Are They Used Consistently?," Journal of Economic Surveys, Wiley Blackwell, vol. 32(1), pages 249-272, February.
    11. Whaples, Robert, 1991. "A Quantitative History of the Journal of Economc History and the Cliometric Revolution," The Journal of Economic History, Cambridge University Press, vol. 51(2), pages 289-301, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    2. Ali Sina Önder & Sergey V. Popov & Sascha Schweitzer, 2021. "Leadership in Scholarship: Editors’ Appointments and the Profession’s Narrative," Working Papers in Economics & Finance 2021-05, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    3. Lorenz Gschwent & Björn Hammarfelt & Martin Karlsson & Mathias Kifmann, 2026. "The Rise of Health Economics: Transforming the Landscape of Economic Research," Health Economics, John Wiley & Sons, Ltd., vol. 35(1), pages 52-68, January.
    4. Pat Pataranutaporn & Nattavudh Powdthavee & Chayapatr Achiwaranguprok & Pattie Maes, 2025. "Can AI Solve the Peer Review Crisis? A Large Scale Cross Model Experiment of LLMs' Performance and Biases in Evaluating over 1000 Economics Papers," Papers 2502.00070, arXiv.org, revised Apr 2025.
    5. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    6. Sebastian Hager & Carlo Schwarz & Fabian Waldinger, 2024. "Measuring Science: Performance Metrics and the Allocation of Talent," American Economic Review, American Economic Association, vol. 114(12), pages 4052-4090, December.
    7. Allen Bellas & Lea-Rachel Kosnik, 2019. "Which leading journal leads? Idea diffusion in economics research journals," Empirical Economics, Springer, vol. 57(3), pages 901-921, September.
    8. Lea†Rachel Kosnik, 2018. "A Survey Of Jel Codes: What Do They Mean And Are They Used Consistently?," Journal of Economic Surveys, Wiley Blackwell, vol. 32(1), pages 249-272, February.
    9. Heikkilä, Jussi T. S., 2024. "Human intelligence versus artificial intelligence in classifying economics research articles: exploratory evidence," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 81(7), pages 18-30.
    10. Andrei Dubovik & Clemens Fiedler & Alexei Parakhonyak, 2022. "Temporal Patterns in Economics Research," CPB Discussion Paper 440, CPB Netherlands Bureau for Economic Policy Analysis.
    11. Josh Angrist & Pierre Azoulay & Glenn Ellison & Ryan Hill & Susan Feng Lu, 2020. "Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship," Journal of Economic Literature, American Economic Association, vol. 58(1), pages 3-52, March.
    12. Michel De Vroey & Luca Pensieroso, 2021. "Grounded in Methodology, Certified by Journals: The Rise and Evolution of a Mainstream in Economics," LIDAM Discussion Papers IRES 2021015, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    13. Lea Kosnik, 2016. "JEL Codes: What Do They Mean and Are They Used Consistently?," Working Papers 1011, University of Missouri-St. Louis, Department of Economics.
    14. Ning Li, 2026. "Merit or networks? What decides where research is published," Papers 2606.03763, arXiv.org.
    15. Spiegel, Yossi & Toivanen, Otto, 2022. "From conference submission to publication and citations: Evidence from the EARIE conference," International Journal of Industrial Organization, Elsevier, vol. 84(C).
    16. Pataranutaporn, Pat & Powdthavee, Nattavudh & Maes, Pattie, 2025. "Can AI Solve the Peer Review Crisis? A Large-Scale Experiment on LLM's Performance and Biases in Evaluating Economics Papers," IZA Discussion Papers 17659, IZA Network @ LISER.
    17. Lutz Bornmann & Klaus Wohlrabe, 2020. "Correction to: Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1167-1167, May.
    18. Saqlain, Muhammad & Merigó, José M. & Gil-Lafuente, Ana M. & Kydland, Finn & Amiguet, Lluis, 2026. "100 volumes of Labour Economics: A bibliometric retrospective," Labour Economics, Elsevier, vol. 100(C).
    19. Gabriela Jaque Herrera & Roberto Cárdenas-Retamal & Daniel Barrales Henriquez, 2022. "Tendencias en Publicaciones en Revistas Chilenas de Economía," Documentos de Trabajo 12, Estudios Nueva Economía.
    20. Lorenzo Ductor & Sanjeev Goyal & Anja Prummer, 2018. "Gender & Collaboration," Working Papers 856, Queen Mary University of London, School of Economics and Finance.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C45 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Neural Networks and Related Topics
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:130163. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.