IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i1d10.1007_s11192-023-04889-x.html
   My bibliography  Save this article

Rescaling the disruption index reveals the universality of disruption distributions in science

Author

Listed:
  • Alex J. Yang

    (Nanjing University
    Jiangsu Key Laboratory of Data Engineering and Knowledge Service
    Nanjing University)

  • Hongcun Gong

    (Nanjing University
    Jiangsu Key Laboratory of Data Engineering and Knowledge Service)

  • Yuhao Wang

    (Nanjing University
    Jiangsu Key Laboratory of Data Engineering and Knowledge Service)

  • Chao Zhang

    (Nanjing University
    Jiangsu Key Laboratory of Data Engineering and Knowledge Service)

  • Sanhong Deng

    (Nanjing University
    Jiangsu Key Laboratory of Data Engineering and Knowledge Service
    Nanjing University)

Abstract

Encompassing an intricately profound propensity for revolutionary, paradigm-shifting ramifications and the potential to wield an irrefutably disruptive sway on forthcoming research endeavors, the notion of the Disruption Index (DI) has surfaced as an object of fervent scientific scrutiny within the realm of scientometrics. Nevertheless, its implementation faces multifaceted constraints. Through a meticulous inquiry, we methodically dissect the limitations of DI, encompassing: (a) susceptibility to variations in reference numbers, (b) vulnerability to intentional author manipulations, (c) heterogeneous manifestations across diverse subject fields, (d) disparities across publication years, (e) misalignment with established scientific impact measures, (f) inadequacy in convergent validity with expert-selected milestones, and (g) a prevalent concentration around zero in its distribution. Unveiling the root causes of these challenges, we propose a viable solution encapsulated in the Rescaled Disruption Index (RDI), achieved through comprehensive rescaling across fields, years, and references. Our empirical investigations unequivocally demonstrate the efficacy of RDI, unveiling the universal nature of disruption distributions in science. This introduces a robust and refined framework for assessing disruptive potential in the scientific landscape while preserving the core principles of the index.

Suggested Citation

  • Alex J. Yang & Hongcun Gong & Yuhao Wang & Chao Zhang & Sanhong Deng, 2024. "Rescaling the disruption index reveals the universality of disruption distributions in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 561-580, January.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04889-x
    DOI: 10.1007/s11192-023-04889-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04889-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04889-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Mokyr, Joel, 1990. "Punctuated Equilibria and Technological Progress," American Economic Review, American Economic Association, vol. 80(2), pages 350-354, May.
    2. Lingfei Wu & Dashun Wang & James A. Evans, 2019. "Large teams develop and small teams disrupt science and technology," Nature, Nature, vol. 566(7744), pages 378-382, February.
    3. Jian Gao & Yi-Cheng Zhang & Tao Zhou, 2019. "Computational Socioeconomics," Papers 1905.06166, arXiv.org.
    4. Betancourt, Nathan & Jochem, Torsten & Otner, Sarah M.G., 2023. "Standing on the shoulders of giants: How star scientists influence their coauthors," Research Policy, Elsevier, vol. 52(1).
    5. Libo Sheng & Dongqing Lyu & Xuanmin Ruan & Hongquan Shen & Ying Cheng, 2023. "The association between prior knowledge and the disruption of an article," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4731-4751, August.
    6. E. Garfield & I. H. Sher, 1963. "New factors in the evaluation of scientific literature through citation indexing," American Documentation, Wiley Blackwell, vol. 14(3), pages 195-201, July.
    7. Joshua Ettinger & Friederike E. L. Otto & E. Lisa F. Schipper, 2021. "Storytelling can be a powerful tool for science," Nature, Nature, vol. 589(7842), pages 352-352, January.
    8. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    9. Sotaro Shibayama & Jian Wang, 2020. "Measuring originality in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 409-427, January.
    10. Michael Park & Erin Leahey & Russell J. Funk, 2023. "Papers and patents are becoming less disruptive over time," Nature, Nature, vol. 613(7942), pages 138-144, January.
    11. Lutz Bornmann & Sitaram Devarakonda & Alexander Tekles & George Chacko, 2020. "Disruptive papers published in Scientometrics: meaningful results by using an improved variant of the disruption index originally proposed by Wu, Wang, and Evans (2019)," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1149-1155, May.
    12. Shalabh, 2021. "Statistical inference via data science," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 1155-1155, July.
    13. Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).
    14. Mariani, Manuel Sebastian & Medo, Matúš & Zhang, Yi-Cheng, 2016. "Identification of milestone papers through time-balanced network centrality," Journal of Informetrics, Elsevier, vol. 10(4), pages 1207-1223.
    15. Yin, Yian & Wang, Dashun, 2017. "The time dimension of science: Connecting the past to the future," Journal of Informetrics, Elsevier, vol. 11(2), pages 608-621.
    16. Arthur, W. Brian, 2007. "The structure of invention," Research Policy, Elsevier, vol. 36(2), pages 274-287, March.
    17. Waltman, Ludo & van Eck, Nees Jan, 2015. "Field-normalized citation impact indicators and the choice of an appropriate counting method," Journal of Informetrics, Elsevier, vol. 9(4), pages 872-894.
    18. Lin, Yiling & Evans, James A. & Wu, Lingfei, 2022. "New directions in science emerge from disconnection and discord," Journal of Informetrics, Elsevier, vol. 16(1).
    19. Mitcham, Carl & Emeritus,, 2021. "Science policy and democracy," Technology in Society, Elsevier, vol. 67(C).
    20. Shouhuai Xu & Moti Yung & Jingguo Wang, 2021. "Seeking Foundations for the Science of Cyber Security," Information Systems Frontiers, Springer, vol. 23(2), pages 263-267, April.
    21. Fontana, Magda & Iori, Martina & Montobbio, Fabio & Sinatra, Roberta, 2020. "New and atypical combinations: An assessment of novelty and interdisciplinarity," Research Policy, Elsevier, vol. 49(7).
    22. Abhay S. D. Rajput & Sangeeta Sharma, 2021. "India: draft science policy calls for public engagement," Nature, Nature, vol. 592(7852), pages 26-26, April.
    23. Wu, Lingfei & Kittur, Aniket & Youn, Hyejin & Milojević, Staša & Leahey, Erin & Fiore, Stephen M. & Ahn, Yong-Yeol, 2022. "Metrics and mechanisms: Measuring the unmeasurable in the science of science," Journal of Informetrics, Elsevier, vol. 16(2).
    24. Leydesdorff, Loet & Bornmann, Lutz, 2021. "Disruption indices and their calculation using web-of-science data: Indicators of historical developments or evolutionary dynamics?," Journal of Informetrics, Elsevier, vol. 15(4).
    25. Chen, Jiyao & Shao, Diana & Fan, Shaokun, 2021. "Destabilization and consolidation: Conceptualizing, measuring, and validating the dual characteristics of technology," Research Policy, Elsevier, vol. 50(1).
    26. Russell J. Funk & Jason Owen-Smith, 2017. "A Dynamic Network Measure of Technological Change," Management Science, INFORMS, vol. 63(3), pages 791-817, March.
    27. Ruan, Xuanmin & Lyu, Dongqing & Gong, Kaile & Cheng, Ying & Li, Jiang, 2021. "Rethinking the disruption index as a measure of scientific and technological advances," Technological Forecasting and Social Change, Elsevier, vol. 172(C).
    28. Jean J. Wang & Sarah X. Shao & Fred Y. Ye, 2021. "Identifying 'seed' papers in sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6001-6011, July.
    29. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    30. Bornmann, Lutz & Tekles, Alexander, 2021. "Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts," Journal of Informetrics, Elsevier, vol. 15(3).
    31. Ludo Waltman & Nees Jan Eck, 2013. "Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 699-716, September.
    32. Nan Deng & An Zeng, 2023. "Enhancing the robustness of the disruption metric against noise," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2419-2428, April.
    33. Kyle Myers, 2020. "The Elasticity of Science," American Economic Journal: Applied Economics, American Economic Association, vol. 12(4), pages 103-134, October.
    34. Johan S. G. Chu & James A. Evans, 2021. "Slowed canonical progress in large fields of science," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(41), pages 2021636118-, October.
    35. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yang, Alex J., 2024. "On the temporal diversity of knowledge in science," Journal of Informetrics, Elsevier, vol. 18(4).
    2. Deng Cheng & Zhang Xue & Yang Zhibo & Zhang Mingze, 2025. "Impact of interdisciplinarity on disruptive innovation: the moderating role of collaboration pattern and collaboration size," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(4), pages 2379-2401, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Li, Meiling & Wang, Yang & Du, Haifeng & Bai, Aruhan, 2024. "Motivating innovation: The impact of prestigious talent funding on junior scientists," Research Policy, Elsevier, vol. 53(9).
    3. Christian Leibel & Lutz Bornmann, 2024. "What do we know about the disruption index in scientometrics? An overview of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 601-639, January.
    4. Zhou, Yuhao & Gong, Faming & Wang, Yanwei & Wang, Ruijie & Zeng, An, 2025. "Fusing structural and temporal information in citation networks for identifying milestone works," Chaos, Solitons & Fractals, Elsevier, vol. 192(C).
    5. Yang, Alex J., 2024. "Unveiling the impact and dual innovation of funded research," Journal of Informetrics, Elsevier, vol. 18(1).
    6. Baicun Li & Aruhan Bai, 2025. "The influence of grant renewal on research content: evidence from NIH-funded PIs," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 2617-2638, May.
    7. Wu, Lingfei & Kittur, Aniket & Youn, Hyejin & Milojević, Staša & Leahey, Erin & Fiore, Stephen M. & Ahn, Yong-Yeol, 2022. "Metrics and mechanisms: Measuring the unmeasurable in the science of science," Journal of Informetrics, Elsevier, vol. 16(2).
    8. Deng Cheng & Zhang Xue & Yang Zhibo & Zhang Mingze, 2025. "Impact of interdisciplinarity on disruptive innovation: the moderating role of collaboration pattern and collaboration size," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(4), pages 2379-2401, April.
    9. Pierre Pelletier & Kevin Wirtz, 2023. "Sails and Anchors: The Complementarity of Exploratory and Exploitative Scientists in Knowledge Creation," Papers 2312.10476, arXiv.org.
    10. Yuyan Jiang & Xueli Liu, 2023. "A construction and empirical research of the journal disruption index based on open citation data," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 3935-3958, July.
    11. Xinhua Chai & Qiang Wu, 2025. "What kind of research network configurations lead to high academic productivity for young management scholars?—A fuzzy-set qualitative comparative analysis (fsQCA)," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 2705-2748, May.
    12. Bornmann, Lutz & Haunschild, Robin, 2024. "The Prize Winner Index (PWI): A proposal for an indicator based on scientific prizes," Journal of Informetrics, Elsevier, vol. 18(4).
    13. Yang, Wenlong & Wang, Yang, 2024. "Exploring team creativity: The nexus between freshness and experience," Journal of Informetrics, Elsevier, vol. 18(4).
    14. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    15. Yue Wang & Ning Li & Bin Zhang & Qian Huang & Jian Wu & Yang Wang, 2023. "The effect of structural holes on producing novel and disruptive research in physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1801-1823, March.
    16. Martin Baily & David Byrne & Aidan Kane & Paul Soto, 2025. "Generative AI at the Crossroads: Light Bulb, Dynamo, or Microscope?," Papers 2505.14588, arXiv.org, revised Jul 2025.
    17. Christian Leibel & Lutz Bornmann, 2024. "Specification uncertainty: what the disruption index tells us about the (hidden) multiverse of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7971-7979, December.
    18. Lutz Bornmann, 2024. "Skewed distributions of scientists’ productivity: a research program for the empirical analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(4), pages 2455-2468, April.
    19. Keye Wu & Ziyue Xie & Jia Tina Du, 2024. "Does science disrupt technology? Examining science intensity, novelty, and recency through patent-paper citations in the pharmaceutical field," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(9), pages 5469-5491, September.
    20. Thomas, Duncan Andrew & Ramos-Vielba, Irene, 2022. "Reframing study of research(er) funding towards configurations and trails," SocArXiv uty2v, Center for Open Science.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04889-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.