IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i17p9515-d620743.html
   My bibliography  Save this article

Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds

Author

Listed:
  • Soo Jeung Lee

    (Department of Education, Sejong University, Seoul 050060, Korea)

  • Christian Schneijderberg

    (International Centre for Higher Education Research (INCHER), University of Kassel, D-34109 Kassel, Germany)

  • Yangson Kim

    (Research Institute for Higher Education (RIHE), Hiroshima University, Higashi-Hiroshima 739-8512, Japan)

  • Isabel Steinhardt

    (International Centre for Higher Education Research (INCHER), University of Kassel, D-34109 Kassel, Germany)

Abstract

Academics may actively respond to the expectations of the academic status market, which have largely been shaped by the World University Rankings (WURs). This study empirically examines how academics’ citation patterns have changed in response to the rise of an “evaluation environment” in academia. We regard the WURs to be a macro-level trigger for cementing a bibliometric-based evaluation environment in academia. Our analyses of citation patterns in papers published in two higher education journals explicitly considered three distinct periods: the pre-WURs (1990–2003), the period of WURs implementation (2004–2010), and the period of adaption to WURs (2011–2017). We applied the nonparametric Kaplan–Meier method to compare first-citation speeds of papers published across the three periods. We found that not only has first-citation speed become faster, but first-citation probability has also increased following the emergence of the WURs. Applying Cox proportional hazard models to first-citation probabilities, we identified journal impact factors and third-party funding as factors influencing first-citation probability, while other author- and paper-related factors showed limited effects. We also found that the general effects of different factors on first-citation speeds have changed with the emergence of the WURs. The findings expand our understanding of the citation patterns of academics in the rise of WURs and provide practical grounds for research policy as well as higher education policy.

Suggested Citation

  • Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
  • Handle: RePEc:gam:jsusta:v:13:y:2021:i:17:p:9515-:d:620743
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/17/9515/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2071-1050/13/17/9515/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jeppe Nicolaisen & Tove Faber Frandsen, 2019. "Zero impact: a large-scale study of uncitedness," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1227-1254, May.
    2. Star X. Zhao & Wen Lou & Alice M. Tan & Shuang Yu, 2018. "Do funded papers attract more usage?," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(1), pages 153-168, April.
    3. Wolfgang Glänzel & Ronald Rousseau & Lin Zhang, 2012. "A visual representation of relative first‐citation times," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(7), pages 1420-1425, July.
    4. Jue Wang & Philip Shapira, 2011. "Funding acknowledgement analysis: an enhanced tool to investigate research sponsorship impacts: the case of nanotechnology," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 563-586, June.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo & Tindaro Cicero, 2012. "What is the appropriate length of the publication period over which to assess research performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1005-1017, December.
    6. Hendrik P. van Dalen & K?ne Henkens, 2005. "Signals in science - On the importance of signaling in gaining attention in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(2), pages 209-233, August.
    7. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    8. Linda Butler, 2003. "Modifying publication practices in response to funding formulas," Research Evaluation, Oxford University Press, vol. 12(1), pages 39-46, April.
    9. Anthony F. J. van Raan, 2004. "Sleeping Beauties in science," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(3), pages 467-472, March.
    10. Laudel, Grit & Gläser, Jochen, 2014. "Beyond breakthrough research: Epistemic properties of research and their consequences for research funding," Research Policy, Elsevier, vol. 43(7), pages 1204-1216.
    11. Dorta-González, Pablo & Dorta-González, María Isabel & Santos-Peñate, Dolores Rosa & Suárez-Vega, Rafael, 2014. "Journal topic citation potential and between-field comparisons: The topic normalized impact factor," Journal of Informetrics, Elsevier, vol. 8(2), pages 406-418.
    12. Wolfgang Glänzel & Ronald Rousseau & Lin Zhang, 2012. "A visual representation of relative first-citation times," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(7), pages 1420-1425, July.
    13. Jian Wang, 2013. "Citation time window choice for research impact evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 851-872, March.
    14. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    15. Agrawal, Ajay & McHale, John & Oettl, Alexander, 2017. "How stars matter: Recruiting and peer effects in evolutionary biology," Research Policy, Elsevier, vol. 46(4), pages 853-867.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    18. Egghe, L. & Bornmann, L. & Guns, R., 2011. "A proposal for a First-Citation-Speed-Index," Journal of Informetrics, Elsevier, vol. 5(1), pages 181-186.
    19. Leo Egghe, 2000. "A Heuristic Study of the First-Citation Distribution," Scientometrics, Springer;Akadémiai Kiadó, vol. 48(3), pages 345-359, July.
    20. Vladimir Pislyakov & Elena Shukshina, 2014. "Measuring excellence in Russia: Highly cited papers, leading institutions, patterns of national and international collaboration," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(11), pages 2321-2330, November.
    21. Bornmann, Lutz & Daniel, Hans-Dieter, 2010. "Citation speed as a measure to predict the attention an article receives: An investigation of the validity of editorial decisions at Angewandte Chemie International Edition," Journal of Informetrics, Elsevier, vol. 4(1), pages 83-88.
    22. Sarah de Rijcke & Paul F. Wouters & Alex D. Rushforth & Thomas P. Franssen & Björn Hammarfelt, 2016. "Evaluation practices and effects of indicator use—a literature review," Research Evaluation, Oxford University Press, vol. 25(2), pages 161-169.
    23. Richard Van Noorden, 2010. "Metrics: A profusion of measures," Nature, Nature, vol. 465(7300), pages 864-866, June.
    24. Elena De la Poza & Paloma Merello & Antonio Barberá & Alberto Celani, 2021. "Universities’ Reporting on SDGs: Using THE Impact Rankings to Model and Measure Their Contribution to Sustainability," Sustainability, MDPI, vol. 13(4), pages 1-28, February.
    25. Xin Gu & Karen Blackmore, 2017. "Quantitative study on Australian academic science," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1009-1035, November.
    26. Jochen Gläser & Grit Laudel, 2005. "Advantages and dangers of ‘remote’ peer evaluation," Research Evaluation, Oxford University Press, vol. 14(3), pages 186-198, December.
    27. Butler, Linda, 2003. "Explaining Australia's increased share of ISI publications--the effects of a funding formula based on publication counts," Research Policy, Elsevier, vol. 32(1), pages 143-155, January.
    28. Peter Woelert & Lachlan McKenzie, 2018. "Follow the money? How Australian universities replicate national performance-based funding mechanisms," Research Evaluation, Oxford University Press, vol. 27(3), pages 184-195.
    29. Henk F. Moed, 2017. "A critical comparative analysis of five world university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 967-990, February.
    30. Rodrigo Costas & Thed N. van Leeuwen & María Bordons, 2012. "Referencing patterns of individual researchers: Do top scientists rely on more extensive information sources?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(12), pages 2433-2450, December.
    31. Jan Youtie, 2014. "The use of citation speed to understand the effects of a multi-institutional science center," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(3), pages 613-621, September.
    32. Mario Cleves & William W. Gould & Roberto G. Gutierrez & Yulia Marchenko, 2010. "An Introduction to Survival Analysis Using Stata," Stata Press books, StataCorp LLC, edition 3, number saus3, July.
    33. Musselin, Christine, 2013. "How peer review empowers the academic profession and university managers: Changes in relationships between the state, universities and the professoriate," Research Policy, Elsevier, vol. 42(5), pages 1165-1173.
    34. Bozeman, Barry & Corley, Elizabeth, 2004. "Scientists' collaboration strategies: implications for scientific and technical human capital," Research Policy, Elsevier, vol. 33(4), pages 599-616, May.
    35. Peder Olesen Larsen & Markus Ins, 2010. "The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 575-603, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sutthisak Srisawad & Kullacha Lertsittiphan & Sirirut Tunsirirut & Pennapa Saenkla, 2025. "Predicting the time to first citation for medical articles using survival analysis and the association between this time and citation counts," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(2), pages 861-880, February.
    2. Guoliang Xu & Longchao Xu & Li Jia, 2022. "Research on Mortality Risk of Chinese Older Adults from the Perspective of Social Health," Sustainability, MDPI, vol. 14(24), pages 1-15, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mingyang Wang & Zhenyu Wang & Guangsheng Chen, 2019. "Which can better predict the future success of articles? Bibliometric indices or alternative metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1575-1595, June.
    2. Pei-Shan Chi & Wolfgang Glänzel, 2024. "Two sides of the same coin? Citation obsolescence and impact of different publication types and subject fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6373-6386, October.
    3. Sánchez-Pérez, Manuel & Marín-Carrillo, María Belén & Segovia-López, Cristina & Terán-Yépez, Eduardo, 2025. "Bibliometric articles in business and management: Factors affecting production and scholarly impact," Journal of Business Research, Elsevier, vol. 186(C).
    4. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    5. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    6. Guoqiang Liang & Haiyan Hou & Xiaodan Lou & Zhigang Hu, 2019. "Qualifying threshold of “take-off” stage for successfully disseminated creative ideas," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1193-1208, September.
    7. Jeppe Nicolaisen & Tove Faber Frandsen, 2021. "Number of references: a large-scale study of interval ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 259-285, January.
    8. Wen-Yau Cathy Lin, 2021. "Effects of open access and articles-in-press mechanisms on publishing lag and first-citation speed: a case on energy and fuels journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4841-4869, June.
    9. Christian Schneijderberg & Nicolai Götze & Lars Müller, 2022. "A study of 25 years of publication outputs in the German academic profession," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 1-28, January.
    10. Liang, Guoqiang & Hou, Haiyan & Ding, Ying & Hu, Zhigang, 2020. "Knowledge recency to the birth of Nobel Prize-winning articles: Gender, career stage, and country," Journal of Informetrics, Elsevier, vol. 14(3).
    11. Groen-Xu, Moqi & Bös, Gregor & Teixeira, Pedro A. & Voigt, Thomas & Knapp, Bernhard, 2023. "Short-term incentives of research evaluations: Evidence from the UK Research Excellence Framework," Research Policy, Elsevier, vol. 52(6).
    12. Anna Tietze & Philip Hofmann, 2019. "The h-index and multi-author hm-index for individual researchers in condensed matter physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 171-185, April.
    13. Wallace, Matthew L. & Ràfols, Ismael, 2018. "Institutional shaping of research priorities: A case study on avian influenza," Research Policy, Elsevier, vol. 47(10), pages 1975-1989.
    14. Gerson Pech & Catarina Delgado, 2020. "Percentile and stochastic-based approach to the comparison of the number of citations of articles indexed in different bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 223-252, April.
    15. Tang, Li & Yang, Defang & Wang, Mingxing & Guo, Ying, 2024. "The mediating impact of citation scope: Evidence from China's ESI publications," Journal of Informetrics, Elsevier, vol. 18(3).
    16. Onodera, Natsuo, 2016. "Properties of an index of citation durability of an article," Journal of Informetrics, Elsevier, vol. 10(4), pages 981-1004.
    17. Broström, Anders, 2019. "Academic breeding grounds: Home department conditions and early career performance of academic researchers," Research Policy, Elsevier, vol. 48(7), pages 1647-1665.
    18. Elizabeth S. Vieira, 2023. "The influence of research collaboration on citation impact: the countries in the European Innovation Scoreboard," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(6), pages 3555-3579, June.
    19. Wang, Jian & Veugelers, Reinhilde & Stephan, Paula, 2017. "Bias against novelty in science: A cautionary tale for users of bibliometric indicators," Research Policy, Elsevier, vol. 46(8), pages 1416-1436.
    20. Libo Sheng & Dongqing Lyu & Xuanmin Ruan & Hongquan Shen & Ying Cheng, 2023. "The association between prior knowledge and the disruption of an article," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(8), pages 4731-4751, August.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:17:p:9515-:d:620743. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.