IDEAS home Printed from https://ideas.repec.org/a/abz/journl/y2025id565.html

Why we don’t need university rankings

Author

Listed:
  • D. M. Kochetkov

Abstract

The proliferation of global university rankings as a tool for research assessment demands critical examination, given the growing volume of methodological criticism and their potentially detrimental impact on academic systems. The aim of this study is to conduct a comprehensive analysis of the inherent problems of rankings and to evaluate current alternatives within the context of shaping science and technology policy. Based on a systematic review of English- and Russian- language academic and grey sources, persistent methodological shortcomings were identified: insufficient transparency of calculation algorithms, limited applicability of bibliometric data for assessing social sciences and humanities (SSH), subjectivity of weighting coefficients in composite indices, and an inability to adequately reflect the quality of the educational processes and managerial efficiency. It was established that rankings relying on a composite indices are fundamentally unsuitable for evaluating organizations due to inherent methodological limitations; rankings without such a composite indicator, based on transparent and reproducible methodology with open access to source data, can be used analytically provided there is strict consideration of the limitations inherent in quantitative metrics. The expediency of transitioning to mission-driven assessment models was substantiated. The significance of the study lies in developing a scientifically grounded position relevant to reforming research assessment systems in Russia and BRICS countries, and in providing arguments for integrating science of science approaches into the practice of science and technology forecasting and governance.

Suggested Citation

  • D. M. Kochetkov, 2025. "Why we don’t need university rankings," Economics of Science, Delo Publishing house, vol. 11(3).
  • Handle: RePEc:abz:journl:y:2025:id:565
    as

    Download full text from publisher

    File URL: https://ecna.elpub.ru/jour/article/viewFile/565/306
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anthony F. J. Raan & Thed N. Leeuwen & Martijn S. Visser, 2011. "Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 495-498, August.
    2. Domingo Docampo & Lawrence Cram, 2014. "On the internal dynamics of the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1347-1366, February.
    3. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    4. Marlo M Vernon & E Andrew Balas & Shaher Momani, 2018. "Are university rankings useful to improve research? A systematic review," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    5. Domingo Docampo, 2011. "On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 77-92, January.
    6. Vicente Safón & Domingo Docampo, 2020. "Analyzing the impact of reputational bias on global university rankings based on objective research performance data: the case of the Shanghai Ranking (ARWU)," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2199-2227, December.
    7. Mu-Hsuan Huang, 2012. "Opening the black box of QS World University Rankings," Research Evaluation, Oxford University Press, vol. 21(1), pages 71-78, February.
    8. Dmitry Kochetkov, 2023. "Review of the Russian-language academic literature on university rankings and a global perspective," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-11, December.
    9. Domingo Docampo, 2011. "Erratum to: On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 237-237, January.
    10. repec:dau:papers:123456789/2947 is not listed on IDEAS
    11. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    2. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    3. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    4. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    5. Milica Jovanovic & Veljko Jeremic & Gordana Savic & Milica Bulajic & Milan Martic, 2012. "How does the normalization of data affect the ARWU ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 319-327, November.
    6. Gul, Muhammet & Yucesan, Melih, 2022. "Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    7. Osmo Kivinen & Juha Hedman & Kalle Artukka, 2017. "Scientific publishing and global university rankings. How well are top publishing universities recognized?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 679-695, July.
    8. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    9. Domingo Docampo & Lawrence Cram, 2015. "On the effects of institutional size in university classifications: the case of the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1325-1346, February.
    10. Jacek Pietrucha, 2018. "Country-specific determinants of world university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1129-1139, March.
    11. Domingo Docampo, 2013. "Reproducibility of the Shanghai academic ranking of world universities results," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 567-587, February.
    12. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    13. Teodoro Luque-Martínez & Salvador Barrio-García, 2016. "Constructing a synthetic indicator of research activity," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1049-1064, September.
    14. Wei-Chao Lin & Ching Chen, 2021. "Novel World University Rankings Combining Academic, Environmental and Resource Indicators," Sustainability, MDPI, vol. 13(24), pages 1-15, December.
    15. Gnewuch, Matthias & Wohlrabe, Klaus, 2018. "Super-efficiency of education institutions: an application to economics departments," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 26, pages 610-623.
    16. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    17. Nicolás Robinson-García & Daniel Torres-Salinas & Emilio Delgado López-Cózar & Francisco Herrera, 2014. "An insight into the importance of national university rankings in an international context: the case of the I-UGR rankings of Spanish universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1309-1324, November.
    18. Barbara Tóth & Hossein Motahari-Nezhad & Nicki Horseman & László Berek & Levente Kovács & Áron Hölgyesi & Márta Péntek & Seyedali Mirjalili & László Gulácsi & Zsombor Zrubka, 2024. "Ranking resilience: assessing the impact of scientific performance and the expansion of the Times Higher Education Word University Rankings on the position of Czech, Hungarian, Polish, and Slovak universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1739-1770, March.
    19. Caterina Cruciani & Silvio Giove & Mehmet Pinar & Matteo Sostero, 2012. "Constructing the FEEM Sustainability Index: A Choquet-Integral Application," Working Papers 2012.50, Fondazione Eni Enrico Mattei.
    20. Marko Marhl & Rene Markovič & Vladimir Grubelnik & Matjaž Perc, 2025. "The changing world dynamics of research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(1), pages 469-488, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:abz:journl:y:2025:id:565. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ð ÐµÐ´Ð°ÐºÑ†Ð¸Ñ (email available below). General contact details of provider: https://delo.ranepa.ru/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.