IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v54y2025i8s0048733325001246.html

Text vs. citations: A comparative analysis of breakthrough and disruption metrics in patent innovation

Author

Listed:
  • Yang, Alex J.

Abstract

This study examines two dynamic metrics for assessing technological innovation— the text-based breakthrough index (KI index) and the citation-based disruption index (CD index)—both of which integrate ex-ante (novelty) and ex-post (impact) information. The KI index identifies breakthrough inventions by measuring their novelty (low similarity to prior patents) and impact (high similarity to future patents), whereas the CD index quantifies technological disruption by analyzing shifts in citation patterns. Using a dataset of over six million patents filed with the USPTO between 1980 and 2017, this paper finds that KI and CD indices are highly correlated and both effectively capture technological breakthroughs. Patents with high KI or CD scores typically originate from original and narrowly focused knowledge bases. However, the two indices exhibit distinct patterns: (1) the KI index fluctuates with economic cycles, while the CD index has experienced a steady decline over time; (2) the KI index positively correlates with future patent citation impact, whereas the CD index follows a U-shaped relationship with patent citation impact; and (3) small and remote teams produce higher KI but lower CD scores, potentially because larger teams cite newer, widely recognized references. I discuss innovation concepts—breakthroughs, disruptions, and beyond—to contextualize these findings and explore their implications for understanding technological advancement. These results contribute to the discourse on measuring innovation and underscore the complementary strengths of text-based and citation-based approaches in assessing technological progress.

Suggested Citation

  • Yang, Alex J., 2025. "Text vs. citations: A comparative analysis of breakthrough and disruption metrics in patent innovation," Research Policy, Elsevier, vol. 54(8).
    • Handle: RePEc:eee:respol:v:54:y:2025:i:8:s0048733325001246
    DOI: 10.1016/j.respol.2025.105295
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733325001246
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2025.105295?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Gabriele Di Bona & Alessandro Bellina & Giordano De Marzo & Angelo Petralia & Iacopo Iacopini & Vito Latora, 2025. "The dynamics of higher-order novelties," Nature Communications, Nature, vol. 16(1), pages 1-12, December.
    2. Michael Park & Erin Leahey & Russell J. Funk, 2023. "Papers and patents are becoming less disruptive over time," Nature, Nature, vol. 613(7942), pages 138-144, January.
    3. Erin Leahey & Jina Lee & Russell J. Funk, 2023. "What Types of Novelty Are Most Disruptive?," American Sociological Review, , vol. 88(3), pages 562-597, June.
    4. Arts, Sam & Hou, Jianan & Gomez, Juan Carlos, 2021. "Natural language processing to identify the creation and impact of new technologies in patent text: Code, data, and new measures," Research Policy, Elsevier, vol. 50(2).
    5. Yiling Lin & Carl Benedikt Frey & Lingfei Wu, 2022. "Remote Collaboration Fuses Fewer Breakthrough Ideas," Papers 2206.01878, arXiv.org, revised Oct 2023.
    6. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    7. Yiling Lin & Carl Benedikt Frey & Lingfei Wu, 2023. "Remote collaboration fuses fewer breakthrough ideas," Nature, Nature, vol. 623(7989), pages 987-991, November.
    8. Strumsky, Deborah & Lobo, José, 2015. "Identifying the sources of technological novelty in the process of invention," Research Policy, Elsevier, vol. 44(8), pages 1445-1461.
    9. Russell J. Funk & Jason Owen-Smith, 2017. "A Dynamic Network Measure of Technological Change," Management Science, INFORMS, vol. 63(3), pages 791-817, March.
    10. Fleming, Lee & Sorenson, Olav, 2001. "Technology as a complex adaptive system: evidence from patent data," Research Policy, Elsevier, vol. 30(7), pages 1019-1039, August.
    11. Bryan Kelly & Dimitris Papanikolaou & Amit Seru & Matt Taddy, 2021. "Measuring Technological Innovation over the Long Run," American Economic Review: Insights, American Economic Association, vol. 3(3), pages 303-320, September.
    12. Johan S. G. Chu & James A. Evans, 2021. "Slowed canonical progress in large fields of science," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(41), pages 2021636118-, October.
    13. Hall, Bronwyn H. & Jaffe, Adam B., 2018. "Measuring Science, Technology, and Innovation: A Review," Annals of Science and Technology Policy, now publishers, vol. 2(1), pages 1-74, March.
    14. Leonid Kogan & Dimitris Papanikolaou & Amit Seru & Noah Stoffman, 2017. "Technological Innovation, Resource Allocation, and Growth," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 132(2), pages 665-712.
    15. Arthur, W. Brian, 2007. "The structure of invention," Research Policy, Elsevier, vol. 36(2), pages 274-287, March.
    16. Fontana, Magda & Iori, Martina & Montobbio, Fabio & Sinatra, Roberta, 2020. "New and atypical combinations: An assessment of novelty and interdisciplinarity," Research Policy, Elsevier, vol. 49(7).
    17. Bas Hofstra & Vivek V. Kulkarni & Sebastian Munoz-Najar Galvez & Bryan He & Dan Jurafsky & Daniel A. McFarland, 2020. "The Diversity–Innovation Paradox in Science," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(17), pages 9284-9291, April.
    18. Higham, Kyle & de Rassenfosse, Gaétan & Jaffe, Adam B., 2021. "Patent Quality: Towards a Systematic Framework for Analysis and Measurement," Research Policy, Elsevier, vol. 50(4).
    19. Lee Fleming, 2001. "Recombinant Uncertainty in Technological Search," Management Science, INFORMS, vol. 47(1), pages 117-132, January.
    20. Mokyr, Joel, 1990. "Punctuated Equilibria and Technological Progress," American Economic Review, American Economic Association, vol. 80(2), pages 350-354, May.
    21. Lingfei Wu & Dashun Wang & James A. Evans, 2019. "Large teams develop and small teams disrupt science and technology," Nature, Nature, vol. 566(7744), pages 378-382, February.
    22. Lin William Cong & Tengyuan Liang & Xiao Zhang & Wu Zhu, 2024. "Textual Factors: A Scalable, Interpretable, and Data-driven Approach to Analyzing Unstructured Information," NBER Working Papers 33168, National Bureau of Economic Research, Inc.
    23. Pierre Azoulay & Joshua S. Graff Zivin & Gustavo Manso, 2011. "Incentives and creativity: evidence from the academic life sciences," RAND Journal of Economics, RAND Corporation, vol. 42(3), pages 527-554, September.
    24. Alex J. Yang & Xiaohui Yan & Haotian Hu & Hanlin Hu & Jia Kong & Sanhong Deng, 2025. "Are disruptive papers more likely to impact technology and society?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 76(3), pages 563-579, March.
    25. Shiyun Wang & Yaxue Ma & Jin Mao & Yun Bai & Zhentao Liang & Gang Li, 2023. "Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(2), pages 150-167, February.
    26. Macher, Jeffrey T. & Rutzer, Christian & Weder, Rolf, 2024. "Is there a secular decline in disruptive patents? Correcting for measurement bias," Research Policy, Elsevier, vol. 53(5).
    27. Sun, Bixuan & Kolesnikov, Sergey & Goldstein, Anna & Chan, Gabriel, 2021. "A dynamic approach for identifying technological breakthroughs with an application in solar photovoltaics," Technological Forecasting and Social Change, Elsevier, vol. 165(C).
    28. Alexander J. Gates & Qing Ke & Onur Varol & Albert-László Barabási, 2019. "Nature’s reach: narrow work has broad impact," Nature, Nature, vol. 575(7781), pages 32-34, November.
    29. Esposito, Christopher R., 2023. "The geography of breakthrough invention in the United States over the 20th century," Research Policy, Elsevier, vol. 52(7).
    30. Christian Leibel & Lutz Bornmann, 2024. "What do we know about the disruption index in scientometrics? An overview of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 601-639, January.
    31. Bronwyn H. Hall & Dietmar Harhoff, 2012. "Recent Research on the Economics of Patents," Annual Review of Economics, Annual Reviews, vol. 4(1), pages 541-565, July.
    32. Alex J. Yang & Hongcun Gong & Yuhao Wang & Chao Zhang & Sanhong Deng, 2024. "Rescaling the disruption index reveals the universality of disruption distributions in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 561-580, January.
    33. Leydesdorff, Loet & Bornmann, Lutz, 2021. "Disruption indices and their calculation using web-of-science data: Indicators of historical developments or evolutionary dynamics?," Journal of Informetrics, Elsevier, vol. 15(4).
    34. Gustavo Manso & Benjamin Balsmeier & Lee Fleming, 2023. "Heterogeneous Innovation over the Business Cycle," The Review of Economics and Statistics, MIT Press, vol. 105(5), pages 1224-1236, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christian Rutzer & Dragan Filimonovic & Jeffrey T. Macher & Rolf Weder, 2026. "Towards Measuring Disruptive Innovation Across Countries," Papers 2603.17881, arXiv.org.
    2. Yongjiao Du & Shi Chen, 2026. "Going for sustainability: how digital transformation affects corporate radical green innovation in China," Economic Change and Restructuring, Springer, vol. 59(1), pages 1-44, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ziyan Zhang & Junyan Zhang & Pushi Wang, 2024. "Measurement of disruptive innovation and its validity based on improved disruption index," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 6477-6531, November.
    2. Yang, Alex J., 2024. "On the temporal diversity of knowledge in science," Journal of Informetrics, Elsevier, vol. 18(4).
    3. Hongkan Chen & Lutz Bornmann & Yi Bu, 2025. "Hot streaks and disruptiveness in the career of scientists: is there an association between both phenomena?," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-15, December.
    4. Alex J. Yang & Hongcun Gong & Yuhao Wang & Chao Zhang & Sanhong Deng, 2024. "Rescaling the disruption index reveals the universality of disruption distributions in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 561-580, January.
    5. Christian Rutzer & Dragan Filimonovic & Jeffrey T. Macher & Rolf Weder, 2026. "Towards Measuring Disruptive Innovation Across Countries," Papers 2603.17881, arXiv.org.
    6. He, Youwei & Lee, Jeong-Dong, 2025. "Small but not least changes: The art of creating disruptive innovations," Journal of Informetrics, Elsevier, vol. 19(3).
    7. Lutz Bornmann & Russell J. Funk, 2025. "Popper’s probability calculus and the decline of scientific disruptiveness," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(8), pages 4801-4807, August.
    8. Sam Arts & Nicola Melluso & Reinhilde Veugelers, 2023. "Beyond Citations: Measuring Novel Scientific Ideas and their Impact in Publication Text," Papers 2309.16437, arXiv.org, revised Dec 2024.
    9. Christian Leibel & Lutz Bornmann, 2024. "Specification uncertainty: what the disruption index tells us about the (hidden) multiverse of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7971-7979, December.
    10. Yuefen Wang & Lipeng Fan & Lei Wu, 2024. "A validation test of the Uzzi et al. novelty measure of innovation and applications to collaboration patterns between institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4379-4394, July.
    11. Yang, Alex J. & Zhang, Yiqin & Wang, Zuorong & Wang, Hao & Deng, Sanhong, 2025. "Quantifying delayed recognition of scientists," Journal of Informetrics, Elsevier, vol. 19(3).
    12. Yang, Alex J., 2024. "Unveiling the impact and dual innovation of funded research," Journal of Informetrics, Elsevier, vol. 18(1).
    13. Li, Meiling & Wang, Yang & Du, Haifeng & Bai, Aruhan, 2024. "Motivating innovation: The impact of prestigious talent funding on junior scientists," Research Policy, Elsevier, vol. 53(9).
    14. Seungmin Lee & Jeong-Dong Lee & Youwei He, 2025. "Exaptation: unveiling the potential for technological innovation," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(7), pages 3483-3503, July.
    15. Higham, Kyle & Contisciani, Martina & De Bacco, Caterina, 2022. "Multilayer patent citation networks: A comprehensive analytical framework for studying explicit technological relationships," Technological Forecasting and Social Change, Elsevier, vol. 179(C).
    16. Yi Zhao & Chengzhi Zhang, 2025. "A review on the novelty measurements of academic papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(2), pages 727-753, February.
    17. Christian Leibel & Lutz Bornmann, 2024. "What do we know about the disruption index in scientometrics? An overview of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 601-639, January.
    18. Wang, Zhongyi & Zhang, Haoxuan & Chen, Jiangping & Chen, Haihua, 2024. "An effective framework for measuring the novelty of scientific articles through integrated topic modeling and cloud model," Journal of Informetrics, Elsevier, vol. 18(4).
    19. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    20. Baicun Li & Aruhan Bai, 2025. "The influence of grant renewal on research content: evidence from NIH-funded PIs," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 2617-2638, May.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes
    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:54:y:2025:i:8:s0048733325001246. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.