IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v87y2011i3d10.1007_s11192-011-0361-6.html
   My bibliography  Save this article

A fresh approach to evaluating the academic ranking of world universities

Author

Listed:
  • Veljko Jeremic

    (University of Belgrade)

  • Milica Bulajic

    (University of Belgrade)

  • Milan Martic

    (University of Belgrade)

  • Zoran Radojicic

    (University of Belgrade)

Abstract

The aim of this article is to present new ideas in evaluating Shanghai University’s Academic Ranking of World Universities (ARWU). One issue frequently put forth in various publications is that the Shanghai rankings are sensitive to the relative weight they attribute to each variable. As a possible remedy to this issue, the statistical I-distance method is proposed to be used. Based on a sample containing the top 100 ranked universities, the results show a significant correlation with the official ARWU list. However, some inconsistencies concerning European universities have been noticed and elaborated upon.

Suggested Citation

  • Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    • Handle: RePEc:spr:scient:v:87:y:2011:i:3:d:10.1007_s11192-011-0361-6
    DOI: 10.1007/s11192-011-0361-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-011-0361-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-011-0361-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Catherine Dehon & Alice McCathie & Vincenzo Verardi, 2010. "Uncovering excellence in academic rankings: a closer look at the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(2), pages 515-524, May.
    2. Domingo Docampo, 2011. "On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 77-92, January.
    3. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    4. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    5. Răzvan V. Florian, 2007. "Irreproducibility of the results of the Shanghai academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 25-32, July.
    6. repec:dau:papers:123456789/2947 is not listed on IDEAS
    7. Isidro F. Aguillo & Judit Bar-Ilan & Mark Levene & José Luis Ortega, 2010. "Comparing university rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 243-256, October.
    8. Jeremic Veljko Milojko & Radojicic Zoran, 2010. "A New Approach in the Evaluation of Team Chess Championships Rankings," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 6(3), pages 1-11, July.
    9. Nian Cai Liu & Ying Cheng & Li Liu, 2005. "Academic ranking of world universities using scientometrics - A comment to the “Fatal Attraction”," Scientometrics, Springer;Akadémiai Kiadó, vol. 64(1), pages 101-109, July.
    10. Domingo Docampo, 2011. "Erratum to: On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 237-237, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Olga Bogdanov & Veljko Jeremiæ & Sandra Jednak & Mladen Èudanov, 2019. "Scrutinizing the Smart City Index: a multivariate statistical approach," Zbornik radova Ekonomskog fakulteta u Rijeci/Proceedings of Rijeka Faculty of Economics, University of Rijeka, Faculty of Economics and Business, vol. 37(2), pages 777-799.
    2. Milica Maricic & Jose A. Egea & Veljko Jeremic, 2019. "A Hybrid Enhanced Scatter Search—Composite I-Distance Indicator (eSS-CIDI) Optimization Approach for Determining Weights Within Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 144(2), pages 497-537, July.
    3. Shahryar Rahnamayan & Sedigheh Mahdavi & Kalyanmoy Deb & Azam Asilian Bidgoli, 2020. "Ranking Multi-Metric Scientific Achievements Using a Concept of Pareto Optimality," Mathematics, MDPI, vol. 8(6), pages 1-46, June.
    4. Milica Jovanovic & Veljko Jeremic & Gordana Savic & Milica Bulajic & Milan Martic, 2012. "How does the normalization of data affect the ARWU ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 319-327, November.
    5. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    6. Nela Miloševiæ & Marina Dobrota & Slaðana Barjaktaroviæ Rakoèeviæ, 2018. "EU enlargement: Digital economy in Europe: Evaluation of countries’ performances," Zbornik radova Ekonomskog fakulteta u Rijeci/Proceedings of Rijeka Faculty of Economics, University of Rijeka, Faculty of Economics and Business, vol. 36(2), pages 861-880.
    7. Milenkovic, Nemanja & Vukmirovic, Jovanka & Bulajic, Milica & Radojicic, Zoran, 2014. "A multivariate approach in measuring socio-economic development of MENA countries," Economic Modelling, Elsevier, vol. 38(C), pages 604-608.
    8. Maričić Milica & Bulajić Milica & Radojičić Zoran & Jeremić Veljko, 2016. "Multivariate approach to imposing additional constraints on the Benefit-of-the-Doubt model: The case of QS World University Rankings by Subject," Croatian Review of Economic, Business and Social Statistics, Sciendo, vol. 2(1), pages 1-14, September.
    9. Lutz Bornmann & Loet Leydesdorff, 2011. "Which cities produce more excellent papers than can be expected? A new mapping approach, using Google Maps, based on statistical significance testing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(10), pages 1954-1962, October.
    10. Gul, Muhammet & Yucesan, Melih, 2022. "Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    11. Hiran H. Lathabai & Abhirup Nandy & Vivek Kumar Singh, 2021. "x-index: Identifying core competency and thematic research strengths of institutions using an NLP and network based ranking framework," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9557-9583, December.
    12. Dobrota, Marina & Martic, Milan & Bulajic, Milica & Jeremic, Veljko, 2015. "Two-phased composite I-distance indicator approach for evaluation of countries’ information development," Telecommunications Policy, Elsevier, vol. 39(5), pages 406-420.
    13. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    14. Maričić Milica & Bulajić Milica & Radojičić Zoran & Jeremić Veljko, 2019. "Shedding Light on the Doing Business Index: a Machine Learning Approach," Business Systems Research, Sciendo, vol. 10(2), pages 73-84, September.
    15. M. Marković & S. Zdravković & M. Mitrović & A. Radojičić, 2016. "An Iterative Multivariate Post Hoc I-Distance Approach in Evaluating OECD Better Life Index," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 126(1), pages 1-19, March.
    16. Aparna Basu & Sumit Kumar Banshal & Khushboo Singhal & Vivek Kumar Singh, 2016. "Designing a Composite Index for research performance evaluation at the national or regional level: ranking Central Universities in India," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1171-1193, June.
    17. Berde Éva & Kuncz Izabella, 2019. "Active Ageing Index, New Emphasis Within the Same Methodology. The Role of the Internet," Studia Universitatis „Vasile Goldis” Arad – Economics Series, Sciendo, vol. 29(4), pages 1-20, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Milica Jovanovic & Veljko Jeremic & Gordana Savic & Milica Bulajic & Milan Martic, 2012. "How does the normalization of data affect the ARWU ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 319-327, November.
    2. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    3. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    4. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    5. Domingo Docampo, 2013. "Reproducibility of the Shanghai academic ranking of world universities results," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 567-587, February.
    6. Alan Peter Matthews, 2012. "South African universities in world rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 675-695, September.
    7. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    8. Gul, Muhammet & Yucesan, Melih, 2022. "Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    9. Osmo Kivinen & Juha Hedman & Kalle Artukka, 2017. "Scientific publishing and global university rankings. How well are top publishing universities recognized?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 679-695, July.
    10. Leo Freyer, 2014. "Robust rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 391-406, August.
    11. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    12. Daniel Torres-Salinas & Jose G. Moreno-Torres & Emilio Delgado-López-Cózar & Francisco Herrera, 2011. "A methodology for Institution-Field ranking based on a bidimensional analysis: the IFQ 2 A index," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 771-786, September.
    13. Domingo Docampo & Lawrence Cram, 2015. "On the effects of institutional size in university classifications: the case of the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1325-1346, February.
    14. Osmo Kivinen & Juha Hedman & Päivi Kaipainen, 2013. "Productivity analysis of research in Natural Sciences, Technology and Clinical Medicine: an input–output model applied in comparison of Top 300 ranked universities of 4 North European and 4 East Asian," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 683-699, February.
    15. El Gibari, Samira & Gómez, Trinidad & Ruiz, Francisco, 2018. "Evaluating university performance using reference point based composite indicators," Journal of Informetrics, Elsevier, vol. 12(4), pages 1235-1250.
    16. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    17. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    18. Abramo, Giovanni & D’Angelo, Ciriaco Andrea, 2015. "Evaluating university research: Same performance indicator, different rankings," Journal of Informetrics, Elsevier, vol. 9(3), pages 514-525.
    19. Enar Ruiz-Conde & Aurora Calderón-Martínez, 2014. "University institutional repositories: competitive environment and their role as communication media of scientific knowledge," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1283-1299, February.
    20. J. A. García & Rosa Rodriguez-Sánchez & J. Fdez-Valdivia & Daniel Torres-Salinas & Francisco Herrera, 2012. "Ranking of research output of universities on the basis of the multidimensional prestige of influential fields: Spanish universities as a case of study," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 1081-1099, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:87:y:2011:i:3:d:10.1007_s11192-011-0361-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.