IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v110y2017i2d10.1007_s11192-016-2212-y.html
   My bibliography  Save this article

A critical comparative analysis of five world university rankings

Author

Listed:
  • Henk F. Moed

    (Sapienza University of Rome)

Abstract

To provide users insight into the value and limits of world university rankings, a comparative analysis is conducted of five ranking systems: ARWU, Leiden, THE, QS and U-Multirank. It links these systems with one another at the level of individual institutions, and analyses the overlap in institutional coverage, geographical coverage, how indicators are calculated from raw data, the skewness of indicator distributions, and statistical correlations between indicators. Four secondary analyses are presented investigating national academic systems and selected pairs of indicators. It is argued that current systems are still one-dimensional in the sense that they provide finalized, seemingly unrelated indicator values rather than offering a dataset and tools to observe patterns in multi-faceted data. By systematically comparing different systems, more insight is provided into how their institutional coverage, rating methods, the selection of indicators and their normalizations influence the ranking positions of given institutions.

Suggested Citation

  • Henk F. Moed, 2017. "A critical comparative analysis of five world university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 967-990, February.
    • Handle: RePEc:spr:scient:v:110:y:2017:i:2:d:10.1007_s11192-016-2212-y
    DOI: 10.1007/s11192-016-2212-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-2212-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-2212-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Paolo Paruolo & Michaela Saisana & Andrea Saltelli, 2013. "Ratings and rankings: voodoo or science?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(3), pages 609-634, June.
    2. Jamil Salmi, 2009. "The Challenge of Establishing World-Class Universities," World Bank Publications - Books, The World Bank Group, number 2600, December.
    3. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    4. Clara Calero-Medina & Carmen López-Illescas & Martijn S Visser & Henk F Moed, 2008. "Important factors when interpreting bibliometric rankings of world universities: an example from oncology," Research Evaluation, Oxford University Press, vol. 17(1), pages 71-81, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Moed, Henk F. & de Moya-Anegón, Félix & López-Illescas, Carmen & Visser, Martijn, 2011. "Is concentration of university research associated with better research performance?," Journal of Informetrics, Elsevier, vol. 5(4), pages 649-658.
    2. Feng Li & Yong Yi & Xiaolong Guo & Wei Qi, 2012. "Performance evaluation of research universities in Mainland China, Hong Kong and Taiwan: based on a two-dimensional approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 531-542, February.
    3. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    4. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    5. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    6. Giovanni Abramo & Ciriaco Andrea D’Angelo & Flavia Di Costa, 2014. "Variability of research performance across disciplines within universities in non-competitive higher education systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 777-795, February.
    7. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    8. Carmen López-Illescas & Félix Moya-Anegón & Henk F. Moed, 2011. "A ranking of universities should account for differences in their disciplinary specialization," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 563-574, August.
    9. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Soldatenkova, Anastasiia, 2016. "The ratio of top scientists to the academic staff as an indicator of the competitive strength of universities," Journal of Informetrics, Elsevier, vol. 10(2), pages 596-605.
    10. Rabishankar Giri & Sabuj Kumar Chaudhuri, 2021. "Ranking journals through the lens of active visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2189-2208, March.
    11. Giovanna Boccuzzo & Licia Maron, 2017. "Proposal of a composite indicator of job quality based on a measure of weighted distances," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(5), pages 2357-2374, September.
    12. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    13. Bjørn Stensaker, 2018. "University Alliances: Enhancing Control, Capacity, and Creativity in Dynamic Environments," Voprosy obrazovaniya / Educational Studies Moscow, National Research University Higher School of Economics, issue 1, pages 132-153.
    14. Osmo Kivinen & Juha Hedman, 2008. "World-wide university rankings: A Scandinavian approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(3), pages 391-408, March.
    15. Giovanni Abramo & Ciriaco Andrea D’Angelo & Marco Solazzi, 2010. "National research assessment exercises: a measure of the distortion of performance rankings when labor input is treated as uniform," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 605-619, September.
    16. Greco, Salvatore & Ishizaka, Alessio & Tasiou, Menelaos & Torrisi, Gianpiero, 2018. "σ-µ efficiency analysis: A new methodology for evaluating units through composite indices," MPRA Paper 83569, University Library of Munich, Germany.
    17. Olavarrieta, Sergio & Quinteros, María José & Villena, Mauricio, 2021. "A Strategic Impact Model for Latin American Business Schools," MPRA Paper 107813, University Library of Munich, Germany, revised 17 Apr 2021.
    18. Khatab Alqararah, 2023. "Assessing the robustness of composite indicators: the case of the Global Innovation Index," Journal of Innovation and Entrepreneurship, Springer, vol. 12(1), pages 1-22, December.
    19. Arega Yirdaw, 2016. "Quality of Education in Private Higher Institutions in Ethiopia," SAGE Open, , vol. 6(1), pages 21582440156, January.
    20. Daraio, Cinzia & Bonaccorsi, Andrea & Geuna, Aldo & Lepori, Benedetto & Bach, Laurent & Bogetoft, Peter & F. Cardoso, Margarida & Castro-Martinez, Elena & Crespi, Gustavo & de Lucio, Ignacio Fernandez, 2011. "The European university landscape: A micro characterization based on evidence from the Aquameth project," Research Policy, Elsevier, vol. 40(1), pages 148-164, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:110:y:2017:i:2:d:10.1007_s11192-016-2212-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.