IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v74y2023i2p150-167.html
   My bibliography  Save this article

Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities

Author

Listed:
  • Shiyun Wang
  • Yaxue Ma
  • Jin Mao
  • Yun Bai
  • Zhentao Liang
  • Gang Li

Abstract

Compared to previous studies that generally detect scientific breakthroughs based on citation patterns, this article proposes a knowledge entity‐based disruption indicator by quantifying the change of knowledge directly created and inspired by scientific breakthroughs to their evolutionary trajectories. Two groups of analytic units, including MeSH terms and their co‐occurrences, are employed independently by the indicator to measure the change of knowledge. The effectiveness of the proposed indicators was evaluated against the four datasets of scientific breakthroughs derived from four recognition trials. In terms of identifying scientific breakthroughs, the proposed disruption indicator based on MeSH co‐occurrences outperforms that based on MeSH terms and three earlier citation‐based disruption indicators. It is also shown that in our indicator, measuring the change of knowledge inspired by the focal paper in its evolutionary trajectory is a larger contributor than measuring the change created by the focal paper. Our study not only offers empirical insights into conceptual understanding of scientific breakthroughs but also provides practical disruption indicator for scientists and science management agencies searching for valuable research.

Suggested Citation

  • Shiyun Wang & Yaxue Ma & Jin Mao & Yun Bai & Zhentao Liang & Gang Li, 2023. "Quantifying scientific breakthroughs by a novel disruption indicator based on knowledge entities," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(2), pages 150-167, February.
    • Handle: RePEc:bla:jinfst:v:74:y:2023:i:2:p:150-167
    DOI: 10.1002/asi.24719
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24719
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24719?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Bryan Kelly & Dimitris Papanikolaou & Amit Seru & Matt Taddy, 2021. "Measuring Technological Innovation over the Long Run," American Economic Review: Insights, American Economic Association, vol. 3(3), pages 303-320, September.
    2. Xue Wang & Xuemei Yang & Jian Du & Xuwen Wang & Jiao Li & Xiaoli Tang, 2021. "A deep learning approach for identifying biomedical breakthrough discoveries using context analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5531-5549, July.
    3. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Benno Torgler, 2013. "Does the John Bates Clark Medal boost subsequent productivity and citation success?," ECON - Working Papers 111, Department of Economics - University of Zurich.
    4. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    5. Russell J. Funk & Jason Owen-Smith, 2017. "A Dynamic Network Measure of Technological Change," Management Science, INFORMS, vol. 63(3), pages 791-817, March.
    6. Guo Zhang & Ying Ding & Staša Milojević, 2013. "Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(7), pages 1490-1503, July.
    7. Petersen, Alexander M. & Pan, Raj K. & Pammolli, Fabio & Fortunato, Santo, 2019. "Methods to account for citation inflation in research evaluation," Research Policy, Elsevier, vol. 48(7), pages 1855-1865.
    8. Min, Chao & Bu, Yi & Sun, Jianjun, 2021. "Predicting scientific breakthroughs based on knowledge structure variations," Technological Forecasting and Social Change, Elsevier, vol. 164(C).
    9. Lingfei Wu & Dashun Wang & James A. Evans, 2019. "Large teams develop and small teams disrupt science and technology," Nature, Nature, vol. 566(7744), pages 378-382, February.
    10. Jesper W. Schneider & Rodrigo Costas, 2017. "Identifying potential “breakthrough” publications using refined citation analyses: Three related explorative approaches," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(3), pages 709-723, March.
    11. Bornmann, Lutz & Tekles, Alexander, 2021. "Convergent validity of several indicators measuring disruptiveness with milestone assignments to physics papers by experts," Journal of Informetrics, Elsevier, vol. 15(3).
    12. Guo Zhang & Ying Ding & Staša Milojević, 2013. "Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(7), pages 1490-1503, July.
    13. Sotaro Shibayama & Deyun Yin & Kuniko Matsumoto, 2021. "Measuring novelty in science with word embedding," PLOS ONE, Public Library of Science, vol. 16(7), pages 1-16, July.
    14. Ponomarev, Ilya V. & Williams, Duane E. & Hackett, Charles J. & Schnell, Joshua D. & Haak, Laurel L., 2014. "Predicting highly cited papers: A Method for Early Detection of Candidate Breakthroughs," Technological Forecasting and Social Change, Elsevier, vol. 81(C), pages 49-55.
    15. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Benno Torgler, 2013. "Does the John Bates Clark Medal boost subsequent productivity and citation success?," ECON - Working Papers 111, Department of Economics - University of Zurich.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Libo Sheng & Dongqing Lyu & Xuanmin Ruan & Hongquan Shen & Ying Cheng, 2023. "The association between prior knowledge and the disruption of an article," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4731-4751, August.
    2. Yuefen Wang & Lipeng Fan & Lei Wu, 2024. "A validation test of the Uzzi et al. novelty measure of innovation and applications to collaboration patterns between institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4379-4394, July.
    3. Yuyan Jiang & Xueli Liu, 2023. "A construction and empirical research of the journal disruption index based on open citation data," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(7), pages 3935-3958, July.
    4. Ziyan Zhang & Junyan Zhang & Pushi Wang, 2024. "Measurement of disruptive innovation and its validity based on improved disruption index," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 6477-6531, November.
    5. Chao Yu & Chuhan Wang & Tongyang Zhang & Yi Bu & Jian Xu, 2024. "Analyzing research diversity of scholars based on multi-dimensional calculation of knowledge entities," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 7329-7358, November.
    6. Baicun Li & Aruhan Bai, 2025. "The influence of grant renewal on research content: evidence from NIH-funded PIs," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 2617-2638, May.
    7. Xin Liu & Yi Bu & Ming Li & Jiang Li, 2024. "Monodisciplinary collaboration disrupts science more than multidisciplinary collaboration," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(1), pages 59-78, January.
    8. Tang, Kun & Li, Baiyang & Zhu, Qiyu & Ma, Lecun, 2024. "Disruptive content, cross agglomeration interaction, and agglomeration replacement: Does cohesion foster strength?," Journal of Informetrics, Elsevier, vol. 18(4).
    9. Jin, Qianqian & Chen, Hongshu & Wang, Xuefeng & Xiong, Fei, 2024. "How do network embeddedness and knowledge stock influence collaboration dynamics? Evidence from patents," Journal of Informetrics, Elsevier, vol. 18(4).
    10. Tong, Tong & Wang, Wanru & Ye, Fred Y., 2024. "A complement to the novel disruption indicator based on knowledge entities," Journal of Informetrics, Elsevier, vol. 18(2).
    11. Zhaoping Yan & Kaiyu Fan, 2024. "An integrated indicator for evaluating scientific papers: considering academic impact and novelty," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(11), pages 6909-6929, November.
    12. Chakraborty, Joyita & Pradhan, Dinesh K. & Nandi, Subrata, 2024. "A multiple k-means cluster ensemble framework for clustering citation trajectories," Journal of Informetrics, Elsevier, vol. 18(2).
    13. Christian Leibel & Lutz Bornmann, 2024. "Specification uncertainty: what the disruption index tells us about the (hidden) multiverse of bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7971-7979, December.
    14. Christian Leibel & Lutz Bornmann, 2024. "What do we know about the disruption index in scientometrics? An overview of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 601-639, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Min, Chao & Bu, Yi & Sun, Jianjun, 2021. "Predicting scientific breakthroughs based on knowledge structure variations," Technological Forecasting and Social Change, Elsevier, vol. 164(C).
    2. Yuefen Wang & Lipeng Fan & Lei Wu, 2024. "A validation test of the Uzzi et al. novelty measure of innovation and applications to collaboration patterns between institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4379-4394, July.
    3. Shiji Chen & Yanan Guo & Alvin Shijie Ding & Yanhui Song, 2024. "Is interdisciplinarity more likely to produce novel or disruptive research?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(5), pages 2615-2632, May.
    4. Zhentao Liang & Jin Mao & Gang Li, 2023. "Bias against scientific novelty: A prepublication perspective," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 99-114, January.
    5. Yue Wang & Ning Li & Bin Zhang & Qian Huang & Jian Wu & Yang Wang, 2023. "The effect of structural holes on producing novel and disruptive research in physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1801-1823, March.
    6. Alex J. Yang & Hongcun Gong & Yuhao Wang & Chao Zhang & Sanhong Deng, 2024. "Rescaling the disruption index reveals the universality of disruption distributions in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 561-580, January.
    7. Jiang, Huihuang & Zhou, Jianlin & Ding, Yiming & Zeng, An, 2024. "Overcoming recognition delays in disruptive research: The impact of team size, familiarity, and reputation," Journal of Informetrics, Elsevier, vol. 18(4).
    8. Hou, Jianhua & Wang, Dongyi & Li, Jing, 2022. "A new method for measuring the originality of academic articles based on knowledge units in semantic networks," Journal of Informetrics, Elsevier, vol. 16(3).
    9. Yi Zhao & Chengzhi Zhang, 2025. "A review on the novelty measurements of academic papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(2), pages 727-753, February.
    10. Christian Leibel & Lutz Bornmann, 2024. "What do we know about the disruption index in scientometrics? An overview of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 601-639, January.
    11. Wang, Zhongyi & Zhang, Haoxuan & Chen, Jiangping & Chen, Haihua, 2024. "An effective framework for measuring the novelty of scientific articles through integrated topic modeling and cloud model," Journal of Informetrics, Elsevier, vol. 18(4).
    12. Dehu Yin & Xi Zhang & Hongke Zhao & Li Tang, 2024. "Predicting scholar potential: a deep learning model on social capital features," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7851-7879, December.
    13. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    14. Li, Xin & Wen, Yang & Jiang, Jiaojiao & Daim, Tugrul & Huang, Lucheng, 2022. "Identifying potential breakthrough research: A machine learning method using scientific papers and Twitter data," Technological Forecasting and Social Change, Elsevier, vol. 184(C).
    15. Baicun Li & Aruhan Bai, 2025. "The influence of grant renewal on research content: evidence from NIH-funded PIs," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 2617-2638, May.
    16. Luo, Zhuoran & Lu, Wei & He, Jiangen & Wang, Yuqi, 2022. "Combination of research questions and methods: A new measurement of scientific novelty," Journal of Informetrics, Elsevier, vol. 16(2).
    17. Julius Koschnick, 2025. "Teacher-directed scientific change:The case of the English Scientific Revolution," Working Papers 0274, European Historical Economics Society (EHES).
    18. Guoqiang Liang & Ying Lou & Haiyan Hou, 2022. "Revisiting the disruptive index: evidence from the Nobel Prize-winning articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5721-5730, October.
    19. He, Guoxiu & Lin, Chenxi & Ren, Jiayu & Duan, Peichen, 2024. "Predicting the emergence of disruptive technologies by comparing with references via soft prompt-aware shared BERT," Journal of Informetrics, Elsevier, vol. 18(4).
    20. Peter Sjögårde & Fereshteh Didegah, 2022. "The association between topic growth and citation impact of research publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1903-1921, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:74:y:2023:i:2:p:150-167. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.