IDEAS home Printed from https://ideas.repec.org/a/eee/tefoso/v123y2017icp153-160.html
   My bibliography  Save this article

Is measuring the knowledge creation of universities possible?: A review of university rankings

Author

Listed:
  • Olcay, Gokcen Arkali
  • Bulu, Melih

Abstract

University ranking indexes are considered very useful benchmarking tools in comparing the performance of universities around the world. Being placed in these prestigious indexes provides a strong advertisement for a university and helps them to attract high-quality students and academicians all over the world. However, there are some important deficiencies of university ranking indexes such as taking into account the whole university as a single unit without differentiating according to different fields of study or research, being limited to some well-known universities, and not considering institutional characteristics such as size or age. This study aims to explore the leading global university rankings to determine the similarities and differences in terms of their ranking criteria, main indicators, modeling choices, and the effects of these on the rankings. Designating the Times Higher Education World Rankings as the base ranking, a comprehensive comparison of the positions of the top universities of the base index with the matched positions of the same universities under other leading indexes including ARWU, QS, Leiden, and URAP is given. Correlations highlight the significant differences among some indexes even in measuring the same criterion such as teaching or research.

Suggested Citation

  • Olcay, Gokcen Arkali & Bulu, Melih, 2017. "Is measuring the knowledge creation of universities possible?: A review of university rankings," Technological Forecasting and Social Change, Elsevier, vol. 123(C), pages 153-160.
    • Handle: RePEc:eee:tefoso:v:123:y:2017:i:c:p:153-160
    DOI: 10.1016/j.techfore.2016.03.029
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S004016251630021X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.techfore.2016.03.029?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Domingo Docampo, 2011. "On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 77-92, January.
    2. Gualberto Buela-Casal & Olga Gutiérrez-Martínez & María Paz Bermúdez-Sánchez & Oscar Vadillo-Muñoz, 2007. "Comparative study of international academic rankings of universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(3), pages 349-365, June.
    3. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    4. Mei Li & Sriram Shankar & Kam Ki Tang, 2009. "Why does the US dominate university league tables?," Discussion Papers Series 391, School of Economics, University of Queensland, Australia.
    5. Osmo Kivinen & Juha Hedman & Päivi Kaipainen, 2013. "Productivity analysis of research in Natural Sciences, Technology and Clinical Medicine: an input–output model applied in comparison of Top 300 ranked universities of 4 North European and 4 East Asian," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 683-699, February.
    6. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    7. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    8. Domingo Docampo, 2011. "Erratum to: On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 237-237, January.
    9. Kuang-hua Chen & Pei-yu Liao, 2012. "A comparative study on world university rankings: a bibliometric survey," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(1), pages 89-103, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Foroudi, Pantea & Yu, Qionglei & Gupta, Suraksha & Foroudi, Mohammad M., 2019. "Enhancing university brand image and reputation through customer value co-creation behaviour," Technological Forecasting and Social Change, Elsevier, vol. 138(C), pages 218-227.
    2. Evgeny V. Balatsky & Natalia A. Ekimova, 2022. "World-class universities and technological development: Unanswered questions," Journal of New Economy, Ural State University of Economics, vol. 23(3), pages 43-61, October.
    3. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2023. "A heuristic approach based on Leiden rankings to identify outliers: evidence from Italian universities in the European landscape," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 483-510, January.
    4. Juan Antonio Dip, 2021. "What does U-multirank tell us about knowledge transfer and research?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3011-3039, April.
    5. Shih-Pin Chen & Chung-Wei Chang, 2021. "Measuring the efficiency of university departments: an empirical study using data envelopment analysis and cluster analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5263-5284, June.
    6. Tatiana Lisitskaya & Pavel Taranov & Ekaterina Ugnich & Vladimir Pislyakov, 2024. "Pillar Universities in Russia: Bibliometrics of ‘the second best’," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(1), pages 365-383, February.
    7. Johan Lyhagen & Per Ahlgren, 2020. "Uncertainty and the ranking of economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2545-2560, December.
    8. Annamaria Demarinis Loiotile & Francesco De Nicolò & Adriana Agrimi & Loredana Bellantuono & Marianna La Rocca & Alfonso Monaco & Ester Pantaleo & Sabina Tangaro & Nicola Amoroso & Roberto Bellotti, 2022. "Best Practices in Knowledge Transfer: Insights from Top Universities," Sustainability, MDPI, vol. 14(22), pages 1-23, November.
    9. Mario González-Sauri & Giulia Rossello, 2023. "The Role of Early-Career University Prestige Stratification on the Future Academic Performance of Scholars," Research in Higher Education, Springer;Association for Institutional Research, vol. 64(1), pages 58-94, February.
    10. Cinzia Daraio & Simone Di Leo & Loet Leydesdorff, 2022. "Using the Leiden Rankings as a Heuristics: Evidence from Italian universities in the European landscape," LEM Papers Series 2022/08, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    11. Csató, László & Tóth, Csaba, 2020. "University rankings from the revealed preferences of the applicants," European Journal of Operational Research, Elsevier, vol. 286(1), pages 309-320.
    12. Adéla Fajčíková & Hana Urbancová, 2019. "Factors Influencing Students’ Motivation to Seek Higher Education—A Case Study at a State University in the Czech Republic," Sustainability, MDPI, vol. 11(17), pages 1-14, August.
    13. Christoph Burmann & Fernando García & Francisco Guijarro & Javier Oliver, 2021. "Ranking the Performance of Universities: The Role of Sustainability," Sustainability, MDPI, vol. 13(23), pages 1-16, November.
    14. Sebastian Aparicio & David Audretsch & David Urbano, 2022. "Governmental Support for Entrepreneurship in Spain: An Institutional Approach," Hacienda Pública Española / Review of Public Economics, IEF, vol. 243(4), pages 29-49, December.
    15. Secundo, Giustina & Rippa, Pierluigi & Cerchione, Roberto, 2020. "Digital Academic Entrepreneurship: A structured literature review and avenue for a research agenda," Technological Forecasting and Social Change, Elsevier, vol. 157(C).
    16. Telcs, András & Kosztyán, Zsolt Tibor & Banász, Zsuzsanna & Csányi, Vivien Valéria, 2019. "Felsőoktatási ligák, parciális rangsorok képzése biklaszterezési eljárásokkal [How to rate higher education systems partial rankings using bi-clustering methods]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(9), pages 905-931.
    17. Guerrero, Maribel & Heaton, Sohvi & Urbano, David, 2021. "Building universities’ intrapreneurial capabilities in the digital era: The role and impacts of Massive Open Online Courses (MOOCs)," Technovation, Elsevier, vol. 99(C).
    18. Zsolt Tibor Kosztyán & Beáta Fehérvölgyi & Tibor Csizmadia & Kinga Kerekes, 2021. "Investigating collaborative and mobility networks: reflections on the core missions of universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3551-3564, April.
    19. Lucas Jódar & Elena De la Poza, 2020. "How and Why the Metric Management Model Is Unsustainable: The Case of Spanish Universities from 2005 to 2020," Sustainability, MDPI, vol. 12(15), pages 1-19, July.
    20. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.
    21. Gabriel Bratucu & Alexandra Palade (Zamfirache) & Anca Madar & Nicoleta Andreea Neacsu, 2017. "Competition on the University Educational Services Market in Romania and the Protection of Students’ Rights and Interests," The AMFITEATRU ECONOMIC journal, Academy of Economic Studies - Bucharest, Romania, vol. 19(45), pages 414-414, May.
    22. Falk, Martin & Hagsten, Eva, 2019. "Attractiveness and efficiency of European universities as hosts for Marie Curie grant holders," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203664, Verein für Socialpolitik / German Economic Association.
    23. Tommaso Agasisti & Guo-liang Yang & Yao-yao Song & Carolyn-Thi Thanh Dung Tran, 2021. "Evaluating the higher education productivity of Chinese and European “elite” universities using a meta-frontier approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5819-5853, July.
    24. Vicente Safón, 2019. "Inter-ranking reputational effects: an analysis of the Academic Ranking of World Universities (ARWU) and the Times Higher Education World University Rankings (THE) reputational relationship," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 897-915, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vicente Safón, 2013. "What do global university rankings really measure? The search for the X factor and the X entity," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 223-244, November.
    2. Osmo Kivinen & Juha Hedman & Kalle Artukka, 2017. "Scientific publishing and global university rankings. How well are top publishing universities recognized?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 679-695, July.
    3. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    4. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    5. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    6. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    7. Gul, Muhammet & Yucesan, Melih, 2022. "Performance evaluation of Turkish Universities by an integrated Bayesian BWM-TOPSIS model," Socio-Economic Planning Sciences, Elsevier, vol. 80(C).
    8. Andrea Bonaccorsi & Cinzia Daraio, 2014. "Beyond university rankings ? Generating new indicators on European universities by linking data in open platforms," DIAG Technical Reports 2014-12, Department of Computer, Control and Management Engineering, Universita' degli Studi di Roma "La Sapienza".
    9. Yang Zhang & Yu Xiao & Jun Wu & Xin Lu, 2021. "Comprehensive world university ranking based on ranking aggregation," Computational Statistics, Springer, vol. 36(2), pages 1139-1152, June.
    10. Teodoro Luque-Martínez & Salvador Barrio-García, 2016. "Constructing a synthetic indicator of research activity," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(3), pages 1049-1064, September.
    11. Domingo Docampo & Lawrence Cram, 2017. "Academic performance and institutional resources: a cross-country analysis of research universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(2), pages 739-764, February.
    12. Bonaccorsi, Andrea & Cicero, Tindaro, 2016. "Nondeterministic ranking of university departments," Journal of Informetrics, Elsevier, vol. 10(1), pages 224-237.
    13. Domingo Docampo & Lawrence Cram, 2015. "On the effects of institutional size in university classifications: the case of the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1325-1346, February.
    14. Pin-Hua Lin & Jong-Rong Chen & Chih-Hai Yang, 2014. "Academic research resources and academic quality: a cross-country analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 109-123, October.
    15. Murat Perit Çakır & Cengiz Acartürk & Oğuzhan Alaşehir & Canan Çilingir, 2015. "A comparative analysis of global and national university ranking systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 813-848, June.
    16. Gnewuch, Matthias & Wohlrabe, Klaus, 2018. "Super-efficiency of education institutions: an application to economics departments," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 26, pages 610-623.
    17. Seiler, Christian & Wohlrabe, Klaus, 2013. "Archetypal scientists," Journal of Informetrics, Elsevier, vol. 7(2), pages 345-356.
    18. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    19. Massucci, Francesco Alessandro & Docampo, Domingo, 2019. "Measuring the academic reputation through citation networks via PageRank," Journal of Informetrics, Elsevier, vol. 13(1), pages 185-201.
    20. Esteban Fernández Tuesta & Máxima Bolaños-Pizarro & Daniel Pimentel Neves & Geziel Fernández & Justin Axel-Berg, 2020. "Complex networks for benchmarking in global universities rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 405-425, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:tefoso:v:123:y:2017:i:c:p:153-160. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.sciencedirect.com/science/journal/00401625 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.