IDEAS home Printed from https://ideas.repec.org/a/abz/journl/y2025id565.html
   My bibliography  Save this article

Why we don’t need university rankings

Author

Listed:
  • D. M. Kochetkov

Abstract

The proliferation of global university rankings as a tool for research assessment demands critical examination, given the growing volume of methodological criticism and their potentially detrimental impact on academic systems. The aim of this study is to conduct a comprehensive analysis of the inherent problems of rankings and to evaluate current alternatives within the context of shaping science and technology policy. Based on a systematic review of English- and Russian- language academic and grey sources, persistent methodological shortcomings were identified: insufficient transparency of calculation algorithms, limited applicability of bibliometric data for assessing social sciences and humanities (SSH), subjectivity of weighting coefficients in composite indices, and an inability to adequately reflect the quality of the educational processes and managerial efficiency. It was established that rankings relying on a composite indices are fundamentally unsuitable for evaluating organizations due to inherent methodological limitations; rankings without such a composite indicator, based on transparent and reproducible methodology with open access to source data, can be used analytically provided there is strict consideration of the limitations inherent in quantitative metrics. The expediency of transitioning to mission-driven assessment models was substantiated. The significance of the study lies in developing a scientifically grounded position relevant to reforming research assessment systems in Russia and BRICS countries, and in providing arguments for integrating science of science approaches into the practice of science and technology forecasting and governance.

Suggested Citation

  • D. M. Kochetkov, 2025. "Why we don’t need university rankings," Economics of Science, Delo Publishing house, vol. 11(3).
  • Handle: RePEc:abz:journl:y:2025:id:565
    as

    Download full text from publisher

    File URL: https://ecna.elpub.ru/jour/article/viewFile/565/306
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    2. repec:dau:papers:123456789/2947 is not listed on IDEAS
    3. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    4. Dmitry Kochetkov, 2023. "Review of the Russian-language academic literature on university rankings and a global perspective," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 10(1), pages 1-11, December.
    5. Anthony F. J. Raan & Thed N. Leeuwen & Martijn S. Visser, 2011. "Severe language effect in university rankings: particularly Germany and France are wronged in citation-based rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 495-498, August.
    6. Mu-Hsuan Huang, 2012. "Opening the black box of QS World University Rankings," Research Evaluation, Oxford University Press, vol. 21(1), pages 71-78, February.
    7. Vicente Safón & Domingo Docampo, 2020. "Analyzing the impact of reputational bias on global university rankings based on objective research performance data: the case of the Shanghai Ranking (ARWU)," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2199-2227, December.
    8. Marlo M Vernon & E Andrew Balas & Shaher Momani, 2018. "Are university rankings useful to improve research? A systematic review," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    2. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    3. Jacek Pietrucha, 2018. "Country-specific determinants of world university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1129-1139, March.
    4. Sung-Shun Weng & Yang Liu & Yen-Ching Chuang, 2019. "Reform of Chinese Universities in the Context of Sustainable Development: Teacher Evaluation and Improvement Based on Hybrid Multiple Criteria Decision-Making Model," Sustainability, MDPI, vol. 11(19), pages 1-23, October.
    5. Alexandr Gedranovich & Mykhaylo Salnykov, 2012. "Productivity analysis of Belarusian higher education system," BEROC Working Paper Series 16, Belarusian Economic Research and Outreach Center (BEROC).
    6. El Gibari, Samira & Gómez, Trinidad & Ruiz, Francisco, 2018. "Evaluating university performance using reference point based composite indicators," Journal of Informetrics, Elsevier, vol. 12(4), pages 1235-1250.
    7. Khatab Alqararah, 2023. "Assessing the robustness of composite indicators: the case of the Global Innovation Index," Journal of Innovation and Entrepreneurship, Springer, vol. 12(1), pages 1-22, December.
    8. Franceschini, Fiorenzo & Maisano, Domenico, 2011. "Structured evaluation of the scientific output of academic research groups by recent h-based indicators," Journal of Informetrics, Elsevier, vol. 5(1), pages 64-74.
    9. Aparna Basu & Sumit Kumar Banshal & Khushboo Singhal & Vivek Kumar Singh, 2016. "Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1171-1193, June.
    10. Telcs, András & Kosztyán, Zsolt Tibor & Banász, Zsuzsanna & Csányi, Vivien Valéria, 2019. "Felsőoktatási ligák, parciális rangsorok képzése biklaszterezési eljárásokkal [How to rate higher education systems partial rankings using bi-clustering methods]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(9), pages 905-931.
    11. Leo Freyer, 2014. "Robust rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 391-406, August.
    12. Zofio, Jose Luis & Aparicio, Juan & Barbero, Javier & Zabala-Iturriagagoitia, Jon Mikel, 2023. "The influence of bottlenecks on innovation systems performance: Put the slowest climber first," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    13. Caterina Cruciani & Silvio Giove & Mehmet Pinar & Matteo Sostero, 2012. "Constructing the FEEM Sustainability Index: A Choquet-Integral Application," Working Papers 2012.50, Fondazione Eni Enrico Mattei.
    14. Giovanni Abramo & Corrado Costa & Ciriaco Andrea D’Angelo, 2015. "A multivariate stochastic model to assess research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1755-1772, February.
    15. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    16. Calabrese, Armando & Capece, Guendalina & Costa, Roberta & Di Pillo, Francesca & Giuffrida, Stefania, 2018. "A ‘power law’ based method to reduce size-related bias in indicators of knowledge performance: An application to university research assessment," Journal of Informetrics, Elsevier, vol. 12(4), pages 1263-1281.
    17. Henk F. Moed & Gali Halevi, 2015. "Multidimensional assessment of scholarly research impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 1988-2002, October.
    18. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    19. Milica Jovanovic & Veljko Jeremic & Gordana Savic & Milica Bulajic & Milan Martic, 2012. "How does the normalization of data affect the ARWU ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 319-327, November.
    20. Franceschini, Fiorenzo & Maisano, Domenico, 2011. "Regularity in the research output of individual scientists: An empirical analysis by recent bibliometric tools," Journal of Informetrics, Elsevier, vol. 5(3), pages 458-468.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:abz:journl:y:2025:id:565. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ð ÐµÐ´Ð°ÐºÑ†Ð¸Ñ (email available below). General contact details of provider: https://delo.ranepa.ru/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.