IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v19y2025i1s1751157724001172.html
   My bibliography  Save this article

The disruption index suffers from citation inflation: Re-analysis of temporal CD trend and relationship with team size reveal discrepancies

Author

Listed:
  • Petersen, Alexander Michael
  • Arroyave, Felber J.
  • Pammolli, Fabio

Abstract

Measuring the rate of innovation in academia and industry is fundamental to monitoring the efficiency and competitiveness of the knowledge economy. To this end, a disruption index (CD) was recently developed and applied to publication and patent citation networks (Wu et al., 2019; Park et al., 2023). Here we show that CD systematically decreases over time due to secular growth in research production, following two distinct mechanisms unrelated to innovation – one behavioral and the other structural. Whereas the behavioral explanation reflects shifts associated with techno-social factors (e.g. self-citation practices), the structural explanation follows from ‘citation inflation’ (CI), an inextricable feature of real citation networks attributable to increasing reference list lengths, which causes CD to systematically decrease. We demonstrate this causal link by way of mathematical deduction, computational simulation, multi-variate regression, and quasi-experimental comparison of the disruptiveness of PNAS versus PNAS Plus articles, which differ primarily in their lengths. Accordingly, we analyze CD data available in the SciSciNet database and find that disruptiveness incrementally increased from 2005-2015, and that the negative relationship between disruption and team-size is remarkably small in overall magnitude effect size, and shifts from negative to positive for team size ≥ 8 coauthors.

Suggested Citation

  • Petersen, Alexander Michael & Arroyave, Felber J. & Pammolli, Fabio, 2025. "The disruption index suffers from citation inflation: Re-analysis of temporal CD trend and relationship with team size reveal discrepancies," Journal of Informetrics, Elsevier, vol. 19(1).
    • Handle: RePEc:eee:infome:v:19:y:2025:i:1:s1751157724001172
    DOI: 10.1016/j.joi.2024.101605
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157724001172
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2024.101605?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Amil Merchant & Simon Batzner & Samuel S. Schoenholz & Muratahan Aykol & Gowoon Cheon & Ekin Dogus Cubuk, 2023. "Scaling deep learning for materials discovery," Nature, Nature, vol. 624(7990), pages 80-85, December.
    2. Lingfei Wu & Dashun Wang & James A. Evans, 2019. "Large teams develop and small teams disrupt science and technology," Nature, Nature, vol. 566(7744), pages 378-382, February.
    3. Li Tang & Philip Shapira & Jan Youtie, 2015. "Is there a clubbing effect underlying Chinese research citation Increases?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1923-1932, September.
    4. Vahe Tshitoyan & John Dagdelen & Leigh Weston & Alexander Dunn & Ziqin Rong & Olga Kononova & Kristin A. Persson & Gerbrand Ceder & Anubhav Jain, 2019. "Unsupervised word embeddings capture latent knowledge from materials science literature," Nature, Nature, vol. 571(7763), pages 95-98, July.
    5. Macher, Jeffrey T. & Rutzer, Christian & Weder, Rolf, 2024. "Is there a secular decline in disruptive patents? Correcting for measurement bias," Research Policy, Elsevier, vol. 53(5).
    6. Michael Park & Erin Leahey & Russell J. Funk, 2023. "Papers and patents are becoming less disruptive over time," Nature, Nature, vol. 613(7942), pages 138-144, January.
    7. Erin Leahey & Jina Lee & Russell J. Funk, 2023. "What Types of Novelty Are Most Disruptive?," American Sociological Review, , vol. 88(3), pages 562-597, June.
    8. Alex Davies & Petar Veličković & Lars Buesing & Sam Blackwell & Daniel Zheng & Nenad Tomašev & Richard Tanburn & Peter Battaglia & Charles Blundell & András Juhász & Marc Lackenby & Geordie Williamson, 2021. "Advancing mathematics by guiding human intuition with AI," Nature, Nature, vol. 600(7887), pages 70-74, December.
    9. R. Alexander Bentley & Sergi Valverde & Joshua Borycz & Blai Vidiella & Benjamin D. Horne & Salva Duran-Nebreda & Michael J. O’Brien, 2023. "Is Disruption Decreasing, Or Is It Accelerating?," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 26(02), pages 1-9, March.
    10. Parolo, Pietro Della Briotta & Pan, Raj Kumar & Ghosh, Rumi & Huberman, Bernardo A. & Kaski, Kimmo & Fortunato, Santo, 2015. "Attention decay in science," Journal of Informetrics, Elsevier, vol. 9(4), pages 734-745.
    11. Josh Abramson & Jonas Adler & Jack Dunger & Richard Evans & Tim Green & Alexander Pritzel & Olaf Ronneberger & Lindsay Willmore & Andrew J. Ballard & Joshua Bambrick & Sebastian W. Bodenstein & David , 2024. "Addendum: Accurate structure prediction of biomolecular interactions with AlphaFold 3," Nature, Nature, vol. 636(8042), pages 4-4, December.
    12. Yiling Lin & Carl Benedikt Frey & Lingfei Wu, 2022. "Remote Collaboration Fuses Fewer Breakthrough Ideas," Papers 2206.01878, arXiv.org, revised Oct 2023.
    13. Max Kozlov, 2023. "‘Disruptive’ science has declined — and no one knows why," Nature, Nature, vol. 613(7943), pages 225-225, January.
    14. Josh Abramson & Jonas Adler & Jack Dunger & Richard Evans & Tim Green & Alexander Pritzel & Olaf Ronneberger & Lindsay Willmore & Andrew J. Ballard & Joshua Bambrick & Sebastian W. Bodenstein & David , 2024. "Accurate structure prediction of biomolecular interactions with AlphaFold 3," Nature, Nature, vol. 630(8016), pages 493-500, June.
    15. Huang, Ying & Chen, Lixin & Zhang, Lin, 2020. "Patent citation inflation: The phenomenon, its measurement, and relative indicators to temper its effects," Journal of Informetrics, Elsevier, vol. 14(2).
    16. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    17. Pan, Raj K. & Petersen, Alexander M. & Pammolli, Fabio & Fortunato, Santo, 2018. "The memory of science: Inflation, myopia, and the knowledge network," Journal of Informetrics, Elsevier, vol. 12(3), pages 656-678.
    18. Hanchen Wang & Tianfan Fu & Yuanqi Du & Wenhao Gao & Kexin Huang & Ziming Liu & Payal Chandak & Shengchao Liu & Peter Katwyk & Andreea Deac & Anima Anandkumar & Karianne Bergen & Carla P. Gomes & Shir, 2023. "Scientific discovery in the age of artificial intelligence," Nature, Nature, vol. 620(7972), pages 47-60, August.
    19. Ruijie Wang & Yuhao Zhou & An Zeng, 2023. "Evaluating scientists by citation and disruption of their representative works," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1689-1710, March.
    20. Yiling Lin & Carl Benedikt Frey & Lingfei Wu, 2023. "Remote collaboration fuses fewer breakthrough ideas," Nature, Nature, vol. 623(7989), pages 987-991, November.
    21. Russell J. Funk & Jason Owen-Smith, 2017. "A Dynamic Network Measure of Technological Change," Management Science, INFORMS, vol. 63(3), pages 791-817, March.
    22. Ruan, Xuanmin & Lyu, Dongqing & Gong, Kaile & Cheng, Ying & Li, Jiang, 2021. "Rethinking the disruption index as a measure of scientific and technological advances," Technological Forecasting and Social Change, Elsevier, vol. 172(C).
    23. Hanchen Wang & Tianfan Fu & Yuanqi Du & Wenhao Gao & Kexin Huang & Ziming Liu & Payal Chandak & Shengchao Liu & Peter Katwyk & Andreea Deac & Anima Anandkumar & Karianne Bergen & Carla P. Gomes & Shir, 2023. "Publisher Correction: Scientific discovery in the age of artificial intelligence," Nature, Nature, vol. 621(7978), pages 33-33, September.
    24. repec:plo:pbio00:3000384 is not listed on IDEAS
    25. Dong Yang & Ioannis Pavlidis & Alexander Michael Petersen, 2023. "Biomedical Convergence Facilitated By The Emergence Of Technological And Informatic Capabilities," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 26(01), pages 1-33, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Deng Cheng & Zhang Xue & Yang Zhibo & Zhang Mingze, 2025. "Impact of interdisciplinarity on disruptive innovation: the moderating role of collaboration pattern and collaboration size," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(4), pages 2379-2401, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christian Leibel & Lutz Bornmann, 2024. "What do we know about the disruption index in scientometrics? An overview of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 601-639, January.
    2. Zhang, Ming-Ze & Wang, Tang-Rong & Lyu, Peng-Hui & Chen, Qi-Mei & Li, Ze-Xia & Ngai, Eric W.T., 2024. "Impact of gender composition of academic teams on disruptive output," Journal of Informetrics, Elsevier, vol. 18(2).
    3. Christian Leibel & Lutz Bornmann, 2024. "Specification uncertainty: what the disruption index tells us about the (hidden) multiverse of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7971-7979, December.
    4. Ziyan Zhang & Junyan Zhang & Pushi Wang, 2024. "Measurement of disruptive innovation and its validity based on improved disruption index," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 6477-6531, November.
    5. Macher, Jeffrey T. & Rutzer, Christian & Weder, Rolf, 2024. "Is there a secular decline in disruptive patents? Correcting for measurement bias," Research Policy, Elsevier, vol. 53(5).
    6. Yang, Alex J., 2024. "On the temporal diversity of knowledge in science," Journal of Informetrics, Elsevier, vol. 18(4).
    7. Ao, Weiyi & Lyu, Dongqing & Ruan, Xuanmin & Li, Jiang & Cheng, Ying, 2023. "Scientific creativity patterns in scholars’ academic careers: Evidence from PubMed," Journal of Informetrics, Elsevier, vol. 17(4).
    8. Sam Arts & Nicola Melluso & Reinhilde Veugelers, 2023. "Beyond Citations: Measuring Novel Scientific Ideas and their Impact in Publication Text," Papers 2309.16437, arXiv.org, revised Dec 2024.
    9. Tang, Kun & Li, Baiyang & Zhu, Qiyu & Ma, Lecun, 2024. "Disruptive content, cross agglomeration interaction, and agglomeration replacement: Does cohesion foster strength?," Journal of Informetrics, Elsevier, vol. 18(4).
    10. Alex J. Yang & Hongcun Gong & Yuhao Wang & Chao Zhang & Sanhong Deng, 2024. "Rescaling the disruption index reveals the universality of disruption distributions in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 561-580, January.
    11. Houqiang Yu & Yian Liang & Yinghua Xie, 2024. "Predicting Scientific Breakthroughs Based on Structural Dynamic of Citation Cascades," Mathematics, MDPI, vol. 12(11), pages 1-18, June.
    12. Jianhua Hou & Bili Zheng & Hao Li & Wenjing Li, 2025. "Evolution and impact of the science of science: from theoretical analysis to digital-AI driven research," Palgrave Communications, Palgrave Macmillan, vol. 12(1), pages 1-9, December.
    13. Yi Zhao & Chengzhi Zhang, 2025. "A review on the novelty measurements of academic papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(2), pages 727-753, February.
    14. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    15. Mingze Zhang & Lili Wang & Zexia Li, 2025. "The impact of team compositions on disruptive and novel research in large-scale research infrastructures," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 2987-3011, May.
    16. Yang, Wenlong & Wang, Yang, 2024. "Exploring team creativity: The nexus between freshness and experience," Journal of Informetrics, Elsevier, vol. 18(4).
    17. Evangelos Katsamakas & Oleg V. Pavlov & Ryan Saklad, 2024. "Artificial intelligence and the transformation of higher education institutions," Papers 2402.08143, arXiv.org.
    18. Wu, Lingfei & Kittur, Aniket & Youn, Hyejin & Milojević, Staša & Leahey, Erin & Fiore, Stephen M. & Ahn, Yong-Yeol, 2022. "Metrics and mechanisms: Measuring the unmeasurable in the science of science," Journal of Informetrics, Elsevier, vol. 16(2).
    19. Chen, Feng & Deng, Hongyu & Zhang, Xiaoying, 2024. "IG-ENT:A innovative ensemble approach for the flow prediction of main steam system in thermal power plant," Energy, Elsevier, vol. 313(C).
    20. Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:19:y:2025:i:1:s1751157724001172. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.