IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/33821.html

Natural Language Processing and Innovation Research

Author

Listed:
  • Antonin Bergeaud
  • Adam B. Jaffe
  • Dimitris Papanikolaou

Abstract

Innovation is central to models in economics, strategy, management, and finance, yet it remains difficult to measure due to its intangible and knowledge-based na ture. Recent advancements in Natural Language Processing offer new methods to analyze textual artifacts, providing empirical insights into previously hard-to-measure aspects of innovation. This paper provides an overview of the current applications of these methods in empirical innovation research, highlights their transformative potential for reshaping how researchers study and quantify innovation, and discusses the critical challenges that accompany their use.

Suggested Citation

  • Antonin Bergeaud & Adam B. Jaffe & Dimitris Papanikolaou, 2025. "Natural Language Processing and Innovation Research," NBER Working Papers 33821, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:33821
    Note: PR
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w33821.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lanjouw, Jean O & Schankerman, Mark, 2001. "Characteristics of Patent Litigation: A Window on Competition," RAND Journal of Economics, The RAND Corporation, vol. 32(1), pages 129-151, Spring.
    2. Argente, David & Baslandze, Salomé & Hanley, Douglas & Moreira, Sara, 2020. "Patents to Products: Product Innovation and Firm Dynamics," CEPR Discussion Papers 14692, Centre for Economic Policy Research.
    3. Flavio Calvino & Chiara Criscuolo & Hélène Dernis & Lea Samek, 2023. "What technologies are at the core of AI?: An exploration based on patent data," OECD Artificial Intelligence Papers 6, OECD Publishing.
    4. Bergeaud Antonin & Verluise Cyril, 2023. "Identifying technology clusters based on automated patent landscaping," PLOS ONE, Public Library of Science, vol. 18(12), pages 1-17, December.
    5. Daron Acemoglu & David Autor & Jonathon Hazell & Pascual Restrepo, 2022. "Artificial Intelligence and Jobs: Evidence from Online Vacancies," Journal of Labor Economics, University of Chicago Press, vol. 40(S1), pages 293-340.
    6. Michael J. Andrews, 2021. "Historical patent data: A practitioner's guide," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 30(2), pages 368-397, May.
    7. Cyril Verluise & Gabriele Cristelli & Kyle Higham & Gaetan de Rassenfosse, 2020. "The Missing 15 Percent of Patent Citations," Working Papers 13, Chair of Science, Technology, and Innovation Policy.
    8. Benjamin F. Jones & Lawrence H. Summers, 2020. "A Calculation of the Social Returns to Innovation," NBER Chapters, in: Innovation and Public Policy, pages 13-59, National Bureau of Economic Research, Inc.
    9. Bergeaud, Antonin & Verluise, Cyril, 2024. "A new dataset to study a century of innovation in Europe and in the US," Research Policy, Elsevier, vol. 53(1).
    10. Robert J. Gordon, 2000. "Does the "New Economy" Measure Up to the Great Inventions of the Past?," Journal of Economic Perspectives, American Economic Association, vol. 14(4), pages 49-74, Fall.
    11. Hall, B. & Jaffe, A. & Trajtenberg, M., 2001. "The NBER Patent Citations Data File: Lessons, Insights and Methodological Tools," Papers 2001-29, Tel Aviv.
    12. Hall, Bronwyn H. & Jaffee, Adam & Trajtenberg, Manuel, 2000. "Market Value and Patent Citations: A First Look," Department of Economics, Working Paper Series qt1rh8k6z2, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
    13. Higham, Kyle & de Rassenfosse, Gaétan & Jaffe, Adam B., 2021. "Patent Quality: Towards a Systematic Framework for Analysis and Measurement," Research Policy, Elsevier, vol. 50(4).
    14. Katja Mann & Lukas Püttmann, 2023. "Benign Effects of Automation: New Evidence from Patent Texts," The Review of Economics and Statistics, MIT Press, vol. 105(3), pages 562-579, May.
    15. Arts, Sam & Hou, Jianan & Gomez, Juan Carlos, 2021. "Natural language processing to identify the creation and impact of new technologies in patent text: Code, data, and new measures," Research Policy, Elsevier, vol. 50(2).
    16. Babina, Tania & Fedyk, Anastassia & He, Alex & Hodson, James, 2024. "Artificial intelligence, firm growth, and product innovation," Journal of Financial Economics, Elsevier, vol. 151(C).
    17. Nicholas Bloom & Charles I. Jones & John Van Reenen & Michael Webb, 2020. "Are Ideas Getting Harder to Find?," American Economic Review, American Economic Association, vol. 110(4), pages 1104-1144, April.
    18. David Lenz & Peter Winker, 2020. "Measuring the diffusion of innovations with paragraph vector topic models," PLOS ONE, Public Library of Science, vol. 15(1), pages 1-18, January.
    19. Aghion, Philippe & Howitt, Peter, 1992. "A Model of Growth through Creative Destruction," Econometrica, Econometric Society, vol. 60(2), pages 323-351, March.
    20. repec:osf:socarx:x78ys_v1 is not listed on IDEAS
    21. Manuel Trajtenberg, 1987. "Patents, Citations and Innovations: Tracing the Links," NBER Working Papers 2457, National Bureau of Economic Research, Inc.
    22. Aharonson, Barak S. & Schilling, Melissa A., 2016. "Mapping the technological landscape: Measuring technology distance, technological footprints, and technology evolution," Research Policy, Elsevier, vol. 45(1), pages 81-96.
    23. Feng Shi & James Evans, 2023. "Surprising combinations of research contents and contexts are related to impact and emerge with scientific outsiders from distant disciplines," Nature Communications, Nature, vol. 14(1), pages 1-13, December.
    24. Matt Marx & Aaron Fuegi, 2020. "Reliance on science: Worldwide front‐page patent citations to scientific articles," Strategic Management Journal, Wiley Blackwell, vol. 41(9), pages 1572-1594, September.
    25. Bryan Kelly & Dimitris Papanikolaou & Amit Seru & Matt Taddy, 2021. "Measuring Technological Innovation over the Long Run," American Economic Review: Insights, American Economic Association, vol. 3(3), pages 303-320, September.
    26. Naomi Hausman, 2022. "University Innovation and Local Economic Growth," The Review of Economics and Statistics, MIT Press, vol. 104(4), pages 718-735, October.
    27. Kriesch, Lukas & Losacker, Sebastian, 2024. "A global patent dataset for the bioeconomy," Papers in Innovation Studies 2024/8, Lund University, CIRCLE - Centre for Innovation Research.
    28. Jeffrey Kuhn & Kenneth Younge & Alan Marco, 2020. "Patent citations reexamined," RAND Journal of Economics, RAND Corporation, vol. 51(1), pages 109-132, March.
    29. Zvi Griliches, 1998. "Patent Statistics as Economic Indicators: A Survey," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 287-343, National Bureau of Economic Research, Inc.
    30. Antonin Bergeaud & Yoann Potiron & Juste Raimbault, 2017. "Classifying patents based on their semantic content," PLOS ONE, Public Library of Science, vol. 12(4), pages 1-22, April.
    31. Joshua Krieger & Danielle Li & Dimitris Papanikolaou, 2022. "Missing Novelty in Drug Development," The Review of Financial Studies, Society for Financial Studies, vol. 35(2), pages 636-679.
    32. Zvi Griliches, 1998. "Issues in Assessing the Contribution of Research and Development to Productivity Growth," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 17-45, National Bureau of Economic Research, Inc.
    33. Romer, Paul M, 1990. "Endogenous Technological Change," Journal of Political Economy, University of Chicago Press, vol. 98(5), pages 71-102, October.
    34. Ariel Pakes & Zvi Griliches, 1980. "Patents and R and D at the Firm Level: A First Look," NBER Working Papers 0561, National Bureau of Economic Research, Inc.
    35. Menaka Hampole & Dimitris Papanikolaou & Lawrence D.W. Schmidt & Bryan Seegmiller, 2025. "Artificial Intelligence and the Labor Market," NBER Working Papers 33509, National Bureau of Economic Research, Inc.
    36. Joshua L. Krieger & Monika Schnitzer & Martin Watzinger, 2024. "Standing on the shoulders of science," Strategic Management Journal, Wiley Blackwell, vol. 45(9), pages 1670-1695, September.
    37. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    38. Pakes, Ariel & Griliches, Zvi, 1980. "Patents and R&D at the firm level: A first report," Economics Letters, Elsevier, vol. 5(4), pages 377-381.
    39. Shawn Kantor & Alexander Whalley, 2014. "Knowledge Spillovers from Research Universities: Evidence from Endowment Value Shocks," The Review of Economics and Statistics, MIT Press, vol. 96(1), pages 171-188, March.
    40. Ufuk Akcigit & Nathan Goldschlag, 2025. "Measuring the characteristics and employment dynamics of U.S. inventors," Journal of Economic Growth, Springer, vol. 30(2), pages 237-269, June.
    41. Jan M. Gerken & Martin G. Moehrle, 2012. "A new instrument for technology monitoring: novelty in patents measured by semantic patent analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 645-670, June.
    42. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    43. Jan Kinne & David Lenz, 2021. "Predicting innovative firms using web mining and deep learning," PLOS ONE, Public Library of Science, vol. 16(4), pages 1-18, April.
    44. de Rassenfosse, Gaétan & Pellegrino, Gabriele & Raiteri, Emilio, 2024. "Do patents enable disclosure? Evidence from the invention secrecy act," International Journal of Industrial Organization, Elsevier, vol. 92(C).
    45. Felix Poege & Dietmar Harhoff & Fabian Gaessler & Stefano Baruffaldi, 2019. "Science Quality and the Value of Inventions," Papers 1903.05020, arXiv.org, revised Apr 2019.
    46. Seegmiller, Bryan & Papanikolaou, Dimitris & Schmidt, Lawrence D.W., 2023. "Measuring document similarity with weighted averages of word embeddings," Explorations in Economic History, Elsevier, vol. 87(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Adam B. Jaffe & Laura B. Shupp & Valentina Tartari, 2026. "Real Effects of Academic Research Revisited," NBER Chapters, in: The Economics of Science: Taking Stock and Looking Ahead, National Bureau of Economic Research, Inc.
    2. Lu, Peng & Mao, Lulu & Zhuang, Xiting & Xu, Zhihong & Ma, Shuai, 2026. "Public perspectives on food date labeling: implications for waste reduction and food safety policy," Food Policy, Elsevier, vol. 140(C).
    3. Grybauskas, Andrius & Vilkas, Mantas & Pilinkien, Vaida & Stundien, Alina & Lukauskas, Mantas, 2026. "Signals of innovation online: Identifying innovative firms by combining website mining and evidence producing LLMs," Technological Forecasting and Social Change, Elsevier, vol. 228(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Arts, Sam & Hou, Jianan & Gomez, Juan Carlos, 2021. "Natural language processing to identify the creation and impact of new technologies in patent text: Code, data, and new measures," Research Policy, Elsevier, vol. 50(2).
    2. Fronzetti Colladon, Andrea & Guardabascio, Barbara & Venturini, Francesco, 2025. "A new mapping of technological interdependence," Research Policy, Elsevier, vol. 54(1).
    3. Bottai, Carlo & de Rassenfosse, Gaétan & Raiteri, Emilio, 2025. "A new approach to measuring invention commercialization: An application to the SBIR program," Research Policy, Elsevier, vol. 54(9).
    4. Higham, Kyle & de Rassenfosse, Gaétan & Jaffe, Adam B., 2021. "Patent Quality: Towards a Systematic Framework for Analysis and Measurement," Research Policy, Elsevier, vol. 50(4).
    5. Mahdiyeh Entezarkheir, 2017. "Patent thickets, defensive patenting, and induced R&D: an empirical analysis of the costs and potential benefits of fragmentation in patent ownership," Empirical Economics, Springer, vol. 52(2), pages 599-634, March.
    6. Bergeaud, Antonin & Schmidt, Julia & Zago, Riccardo, 2026. "Patents that match your standards: Firm-level evidence on competition, innovation and growth," Journal of Financial Economics, Elsevier, vol. 180(C).
    7. Carlino, Gerald & Kerr, William R., 2015. "Agglomeration and Innovation," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 349-404, Elsevier.
    8. Antonin Bergeaud & Julia Schmidt & Riccardo Zago, 2022. "Patents that Match your Standards: Firm-level Evidence on Competition and Growth," Working papers 876, Banque de France.
    9. Ufuk Akcigit & William R. Kerr, 2018. "Growth through Heterogeneous Innovations," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1374-1443.
    10. Baghai, Ramin & Silva, Rui & Ye, Luofu, 2018. "Teams and Bankruptcy," CEPR Discussion Papers 13198, Centre for Economic Policy Research.
    11. repec:bof:bofrdp:urn:nbn:fi:bof-201512111472 is not listed on IDEAS
    12. Gao, Wenlian & Chou, Julia, 2015. "Innovation efficiency, global diversification, and firm value," Journal of Corporate Finance, Elsevier, vol. 30(C), pages 278-298.
    13. Denter, Nils M. & Waterstraat, Joe & Moehrle, Martin G., 2025. "Avoiding the pitfalls of direct linkage: A novelty-driven approach to measuring scientific impact on patents," Journal of Informetrics, Elsevier, vol. 19(2).
    14. Ufuk Akcigit & William R. Kerr, 2018. "Growth through Heterogeneous Innovations," Journal of Political Economy, University of Chicago Press, vol. 126(4), pages 1374-1443.
    15. Hain, Daniel S. & Jurowetzki, Roman & Buchmann, Tobias & Wolf, Patrick, 2022. "A text-embedding-based approach to measuring patent-to-patent technological similarity," Technological Forecasting and Social Change, Elsevier, vol. 177(C).
    16. Enrico Berkes & Peter Nencka, 2026. "Knowledge Access: The Effects of Carnegie Libraries on Innovation," The Review of Economics and Statistics, MIT Press, vol. 108(4), pages 967-982, July.
    17. Teresa C. Fort & Nathan Goldschlag & Jack Liang & Peter K. Schott & Nikolas Zolas, 2025. "Growth is Getting Harder to Find, Not Ideas," Working Papers 25-21, Center for Economic Studies, U.S. Census Bureau.
    18. Carlino, Gerald & Kerr, William R., 2015. "Agglomeration and Innovation," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 349-404, Elsevier.
    19. Stephan, Annegret & Bening, Catharina R. & Schmidt, Tobias S. & Schwarz, Marius & Hoffmann, Volker H., 2019. "The role of inter-sectoral knowledge spillovers in technological innovations: The case of lithium-ion batteries," Technological Forecasting and Social Change, Elsevier, vol. 148(C).
    20. Bergeaud, Antonin & Schmidt, Julia & Zago, Riccardo, 2026. "Innovation, Technology Standardization and the Value of the Firm," HEC Research Papers Series 1613, HEC Paris.
    21. Sitian Liu & Yichen Su, 2026. "The Geography of Research: The Trade-Off Between Knowledge Production and Access," Papers 2608.15981, arXiv.org.

    More about this item

    JEL classification:

    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General
    • O30 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - General
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:33821. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.