IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v130y2025i2d10.1007_s11192-024-05158-1.html
   My bibliography  Save this article

The definition of highly cited researchers: the effect of different approaches on the empirical outcome

Author

Listed:
  • Rainer Frietsch

    (Fraunhofer Institute for Systems and Innovation Research)

  • Sonia Gruber

    (Fraunhofer Institute for Systems and Innovation Research)

  • Lutz Bornmann

    (Administrative Headquarters of the Max Planck Society)

Abstract

In 2001 onetime and since 2014 annually, Clarivate (and the former Thomson Reuters) has used publication and citation data to identify exceptional researchers—highly cited researchers (HCRs)—in nearly all disciplines. The approach used by Clarivate has not been without criticism. HCRs can be defined differently; the approach of Clarivate is one possibility among several others. HCRs can be identified by considering field-normalized citation rates or absolute numbers of citations; inclusion or exclusion of self-citations; full counting or fractional counting of publications; all authors, only corresponding authors or only first authors; short, long or varying citation windows; and short or long publication periods. In this study, we are interested in the effect different approaches have on the empirical outcomes. One may expect HCRs lists with large overlaps of authors, since all approaches are based on the same (bibliometric) data. As we demonstrated with five different variants of defining HCRs, the selection among these options has a significant influence on the sample of selected researchers and their characteristics that are thereby defined as highly cited. Some options have a stronger influence on the outcome than other options such as the length of the citation window or the focus on all authors versus only the corresponding author. Based on the empirical results of this study, we recommend that the user of HCR lists should always be aware of the influence these options have on the final lists of researchers.

Suggested Citation

  • Rainer Frietsch & Sonia Gruber & Lutz Bornmann, 2025. "The definition of highly cited researchers: the effect of different approaches on the empirical outcome," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(2), pages 881-907, February.
    • Handle: RePEc:spr:scient:v:130:y:2025:i:2:d:10.1007_s11192-024-05158-1
    DOI: 10.1007/s11192-024-05158-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05158-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05158-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Joshua Ettinger & Friederike E. L. Otto & E. Lisa F. Schipper, 2021. "Storytelling can be a powerful tool for science," Nature, Nature, vol. 589(7842), pages 352-352, January.
    2. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    3. Valeria Aman, 2018. "Does the Scopus author ID suffice to track scientific international mobility? A case study based on Leibniz laureates," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 705-720, November.
    4. Hanna-Mari Puuska & Reetta Muhonen & Yrjö Leino, 2014. "International and domestic co-publishing and their citation impact in different disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 823-839, February.
    5. Luc Bauwens & Giordano Mion & Jacques-François Thisse, 2011. "The Resistible Decline of European Science," Recherches économiques de Louvain, De Boeck Université, vol. 77(4), pages 5-31.
    6. John T. Li, 2016. "What we learn from the shifts in highly cited data from 2001 to 2014?," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(1), pages 57-82, July.
    7. Nabil Amara & Réjean Landry & Norrin Halilem, 2015. "What can university administrators do to increase the publication and citation scores of their faculty members?," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 489-530, May.
    8. Alexandra-Maria Klein & Nina Kranke, 2023. "Some thoughts on transparency of the data and analysis behind the Highly Cited Researchers list," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(12), pages 6773-6780, December.
    9. John P A Ioannidis & Kevin W Boyack & Richard Klavans, 2014. "Estimates of the Continuously Publishing Core in the Scientific Workforce," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-10, July.
    10. Henk F. Moed & M’hamed Aisati & Andrew Plume, 2013. "Studying scientific migration in Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 929-942, March.
    11. Carolin Michels & Ulrich Schmoch, 2014. "Impact of bibliometric studies on the publication behaviour of authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 369-385, January.
    12. Mitcham, Carl & Emeritus,, 2021. "Science policy and democracy," Technology in Society, Elsevier, vol. 67(C).
    13. Shouhuai Xu & Moti Yung & Jingguo Wang, 2021. "Seeking Foundations for the Science of Cyber Security," Information Systems Frontiers, Springer, vol. 23(2), pages 263-267, April.
    14. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    15. Ulrich Schmoch & Torben Schubert, 2008. "Are international co-publications an indicator for quality of scientific research?," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(3), pages 361-377, March.
    16. Michels, Carolin & Fu, Junying, 2013. "Systematic analysis of coverage and usage of conference proceedings in web of science: Subproject in the Kompetenzzentrum Bibliometrie," Discussion Papers "Innovation Systems and Policy Analysis" 33, Fraunhofer Institute for Systems and Innovation Research (ISI).
    17. Giovanni Abramo & Ciriaco Andrea D’Angelo & Anastasiia Soldatenkova, 2017. "How long do top scientists maintain their stardom? An analysis by region, gender and discipline: evidence from Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 867-877, February.
    18. Giovanni Abramo & Ciriaco Andrea D’Angelo & Anastasiia Soldatenkova, 2017. "Erratum to: How long do top scientists maintain their stardom? An analysis by region, gender and discipline: evidence from Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1825-1825, December.
    19. Lutz Bornmann & Johann Bauer & Robin Haunschild, 2015. "Distribution of women and men among highly cited scientists," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2715-2716, December.
    20. repec:plo:pbio00:3000384 is not listed on IDEAS
    21. repec:plo:pbio00:1002501 is not listed on IDEAS
    22. John P A Ioannidis & Kevin W Boyack & Jeroen Baas, 2020. "Updated science-wide author databases of standardized citation indicators," PLOS Biology, Public Library of Science, vol. 18(10), pages 1-3, October.
    23. Mike Thelwall & Nabeil Maflahi, 2020. "Academic collaboration rates and citation associations vary substantially between countries and fields," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 71(8), pages 968-978, August.
    24. Weihua Li & Tomaso Aste & Fabio Caccioli & Giacomo Livan, 2019. "Early coauthorship with top scientists predicts success in academic careers," Nature Communications, Nature, vol. 10(1), pages 1-9, December.
    25. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
    26. Shalabh, 2021. "Statistical inference via data science," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 1155-1155, July.
    27. Teja Tscharntke & Michael E Hochberg & Tatyana A Rand & Vincent H Resh & Jochen Krauss, 2007. "Author Sequence and Credit for Contributions in Multiauthored Publications," PLOS Biology, Public Library of Science, vol. 5(1), pages 1-2, January.
    28. Lutz Bornmann & Johann Bauer, 2015. "Evaluation of the highly-cited researchers’ database for a country: proposals for meaningful analyses on the example of Germany," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1997-2003, December.
    29. Abhay S. D. Rajput & Sangeeta Sharma, 2021. "India: draft science policy calls for public engagement," Nature, Nature, vol. 592(7852), pages 26-26, April.
    30. Lisa Mandle & Analisa Shields-Estrada & Rebecca Chaplin-Kramer & Matthew G. E. Mitchell & Leah L. Bremer & Jesse D. Gourevitch & Peter Hawthorne & Justin A. Johnson & Brian E. Robinson & Jeffrey R. Sm, 2021. "Increasing decision relevance of ecosystem service science," Nature Sustainability, Nature, vol. 4(2), pages 161-169, February.
    31. Jean J. Wang & Sarah X. Shao & Fred Y. Ye, 2021. "Identifying 'seed' papers in sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6001-6011, July.
    32. repec:plo:pone00:0196341 is not listed on IDEAS
    33. Félix Moya-Anegón & Vicente P. Guerrero-Bote & Lutz Bornmann & Henk F. Moed, 2013. "The research guarantors of scientific papers and the output counting: a promising new approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 421-434, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lutz Bornmann, 2024. "Skewed distributions of scientists’ productivity: a research program for the empirical analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(4), pages 2455-2468, April.
    2. Liang, Zhentao & Ba, Zhichao & Mao, Jin & Li, Gang, 2023. "Research complexity increases with scientists’ academic age: Evidence from library and information science," Journal of Informetrics, Elsevier, vol. 17(1).
    3. Weihua Li & Sam Zhang & Zhiming Zheng & Skyler J. Cranmer & Aaron Clauset, 2022. "Untangling the network effects of productivity and prominence among scientists," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    4. Bornmann, Lutz & Haunschild, Robin, 2024. "The Prize Winner Index (PWI): A proposal for an indicator based on scientific prizes," Journal of Informetrics, Elsevier, vol. 18(4).
    5. Malte Hückstädt, 2023. "Ten reasons why research collaborations succeed—a random forest approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1923-1950, March.
    6. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    7. Martin Baily & David Byrne & Aidan Kane & Paul Soto, 2025. "Generative AI at the Crossroads: Light Bulb, Dynamo, or Microscope?," Papers 2505.14588, arXiv.org, revised Jun 2025.
    8. Liyin Zhang & Yuchen Qian & Chao Ma & Jiang Li, 2023. "Continued collaboration shortens the transition period of scientists who move to another institution," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1765-1784, March.
    9. Michael Färber & Melissa Coutinho & Shuzhou Yuan, 2023. "Biases in scholarly recommender systems: impact, prevalence, and mitigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2703-2736, May.
    10. Marek Kwiek & Wojciech Roszka, 2022. "Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3543-3575, June.
    11. Alex J. Yang & Hongcun Gong & Yuhao Wang & Chao Zhang & Sanhong Deng, 2024. "Rescaling the disruption index reveals the universality of disruption distributions in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 561-580, January.
    12. Zhuanlan Sun & C. Clark Cao & Sheng Liu & Yiwei Li & Chao Ma, 2024. "Behavioral consequences of second-person pronouns in written communications between authors and reviewers of scientific papers," Nature Communications, Nature, vol. 15(1), pages 1-12, December.
    13. Thomas, Duncan Andrew & Ramos-Vielba, Irene, 2022. "Reframing study of research(er) funding towards configurations and trails," SocArXiv uty2v, Center for Open Science.
    14. Jiang, Zhuoren & Lin, Tianqianjin & Huang, Cui, 2023. "Deep representation learning of scientific paper reveals its potential scholarly impact," Journal of Informetrics, Elsevier, vol. 17(1).
    15. Peng, Xianzhe & Xu, Huixin & Shi, Jin, 2024. "Are the bibliometric growth patterns of excellent scholars similar? From the analysis of ACM Fellows," Journal of Informetrics, Elsevier, vol. 18(3).
    16. Pedro Pessoa & Steve Pressé, 2023. "How many submissions are needed to discover friendly suggested reviewers?," PLOS ONE, Public Library of Science, vol. 18(4), pages 1-17, April.
    17. Yang, Wenlong & Wang, Yang, 2024. "Exploring team creativity: The nexus between freshness and experience," Journal of Informetrics, Elsevier, vol. 18(4).
    18. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2023. "A heuristic approach based on Leiden rankings to identify outliers: evidence from Italian universities in the European landscape," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 483-510, January.
    19. repec:osf:socarx:aep9v_v1 is not listed on IDEAS
    20. Martin Mende & David Glen Mick, 2024. "A commentary on transformative consumer research: Musings on its genesis, evolution, and opportunity for scientific specialization," AMS Review, Springer;Academy of Marketing Science, vol. 14(3), pages 182-193, December.
    21. Li, Meiling & Wang, Yang & Du, Haifeng & Bai, Aruhan, 2024. "Motivating innovation: The impact of prestigious talent funding on junior scientists," Research Policy, Elsevier, vol. 53(9).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:130:y:2025:i:2:d:10.1007_s11192-024-05158-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.