IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v72y2007i1d10.1007_s11192-007-1712-1.html
   My bibliography  Save this article

Irreproducibility of the results of the Shanghai academic ranking of world universities

Author

Listed:
  • Răzvan V. Florian

    (Ad Astra Association of Romanian Scientists
    Center for Cognitive and Neural Studies (Coneural))

Abstract

I discuss the difficulties that I encountered in reproducing the results of the Shanghai ranking of world universities. In the Shanghai ranking, the dependence between the score for the SCI indicator and the weighted number of considered articles obeys a power law, instead of the proportional dependence that is suggested by the official methodology of the ranking. Discrepancies from proportionality are also found in some of the scores for the N&S and Size indicators. This shows that the results of the Shanghai ranking cannot be reproduced, given raw data and the public methodology of the ranking.

Suggested Citation

  • Răzvan V. Florian, 2007. "Irreproducibility of the results of the Shanghai academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(1), pages 25-32, July.
    • Handle: RePEc:spr:scient:v:72:y:2007:i:1:d:10.1007_s11192-007-1712-1
    DOI: 10.1007/s11192-007-1712-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-007-1712-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-007-1712-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    2. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    3. Ludo Waltman & Clara Calero-Medina & Joost Kosten & Ed C.M. Noyons & Robert J.W. Tijssen & Nees Jan Eck & Thed N. Leeuwen & Anthony F.J. Raan & Martijn S. Visser & Paul Wouters, 2012. "The Leiden ranking 2011/2012: Data collection, indicators, and interpretation," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(12), pages 2419-2432, December.
    4. Lucia Buzzigoli & Antonio Giusti & Alessandro Viviani, 2010. "The Evaluation of University Departments. A Case Study for Firenze," International Advances in Economic Research, Springer;International Atlantic Economic Society, vol. 16(1), pages 24-38, February.
    5. Ulrich Schmoch & Habib M. Fardoun & Abdulfattah S. Mashat, 2016. "Establishing a World-Class University in Saudi Arabia: intended and unintended effects," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1191-1207, November.
    6. Paul F. Skilton, 2009. "Does the human capital of teams of natural science authors predict citation frequency?," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(3), pages 525-542, March.
    7. Domingo Docampo, 2013. "Reproducibility of the Shanghai academic ranking of world universities results," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 567-587, February.
    8. Poh Kam Wong & Annette Singh, 2010. "University patenting activities and their link to the quantity and quality of scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 271-294, April.
    9. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2010. "Should you believe in the Shanghai ranking?," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 237-263, July.
    10. Osmo Kivinen & Juha Hedman & Päivi Kaipainen, 2013. "Productivity analysis of research in Natural Sciences, Technology and Clinical Medicine: an input–output model applied in comparison of Top 300 ranked universities of 4 North European and 4 East Asian," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 683-699, February.
    11. Leo Freyer, 2014. "Robust rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 391-406, August.
    12. Jean-Charles Billaut & Denis Bouyssou & Philippe Vincke, 2009. "Should you believe in the Shanghai ranking? An MCDM view," Working Papers hal-00877050, HAL.
    13. Enis Siniksaran & M. Hakan Satman, 2020. "WURS: a simulation software for university rankings—software review," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 701-717, January.
    14. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    15. Daniele Fanelli, 2010. "Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data," PLOS ONE, Public Library of Science, vol. 5(4), pages 1-7, April.
    16. Billaut, Jean-Charles & Bouyssou, Denis & Vincke, Philippe, 2010. "Faut-il croire le classement de Shangaï ?," Revue de la Régulation - Capitalisme, institutions, pouvoirs, Association Recherche et Régulation, vol. 8.
    17. Thum, Anna-Elisabeth & Roth,Felix, 2010. "The Key Role of Education in the Europe 2020 Strategy," CEPS Papers 3827, Centre for European Policy Studies.
    18. Antonio Fernández-Cano & Elvira Curiel-Marin & Manuel Torralbo-Rodríguez & Mónica Vallejo-Ruiz, 2018. "Questioning the Shanghai Ranking methodology as a tool for the evaluation of universities: an integrative review," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2069-2083, September.
    19. Domingo Docampo, 2011. "On using the Shanghai ranking to assess the research performance of university systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(1), pages 77-92, January.
    20. Veljko Jeremic & Milica Bulajic & Milan Martic & Zoran Radojicic, 2011. "A fresh approach to evaluating the academic ranking of world universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 587-596, June.
    21. Viktor Koksharov, 2014. "Development Dynamics of the Higher Education System in Russia," Economy of region, Centre for Economic Security, Institute of Economics of Ural Branch of Russian Academy of Sciences, vol. 1(4), pages 30-44.
    22. Viktor Koksharov & Daniil Sandler & Sergey Kadochnikov & Dmitrii Tolmachev, 2012. "Research and development potential of the Russian higher education institutions," Economy of region, Centre for Economic Security, Institute of Economics of Ural Branch of Russian Academy of Sciences, vol. 1(3), pages 33-46.
    23. O. Mryglod & R. Kenna & Yu. Holovatch & B. Berche, 2013. "Comparison of a citation-based indicator and peer review for absolute and specific measures of research-group excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 767-777, December.
    24. Alan Peter Matthews, 2012. "South African universities in world rankings," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 675-695, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rabishankar Giri & Sabuj Kumar Chaudhuri, 2021. "Ranking journals through the lens of active visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2189-2208, March.
    2. Domingo Docampo & Lawrence Cram, 2019. "Highly cited researchers: a moving target," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(3), pages 1011-1025, March.
    3. Osmo Kivinen & Juha Hedman, 2008. "World-wide university rankings: A Scandinavian approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(3), pages 391-408, March.
    4. Feng Li & Yong Yi & Xiaolong Guo & Wei Qi, 2012. "Performance evaluation of research universities in Mainland China, Hong Kong and Taiwan: based on a two-dimensional approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 531-542, February.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo & Marco Solazzi, 2010. "National research assessment exercises: a measure of the distortion of performance rankings when labor input is treated as uniform," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 605-619, September.
    6. Daraio, Cinzia & Bonaccorsi, Andrea & Geuna, Aldo & Lepori, Benedetto & Bach, Laurent & Bogetoft, Peter & F. Cardoso, Margarida & Castro-Martinez, Elena & Crespi, Gustavo & de Lucio, Ignacio Fernandez, 2011. "The European university landscape: A micro characterization based on evidence from the Aquameth project," Research Policy, Elsevier, vol. 40(1), pages 148-164, February.
    7. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    8. Daraio, Cinzia & Bonaccorsi, Andrea & Simar, Léopold, 2015. "Rankings and university performance: A conditional multidimensional approach," European Journal of Operational Research, Elsevier, vol. 244(3), pages 918-930.
    9. Giovanni Abramo & Ciriaco Andrea D’Angelo & Marco Solazzi, 2011. "The relationship between scientists’ research performance and the degree of internationalization of their research," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(3), pages 629-643, March.
    10. Power, Michael, 2015. "How accounting begins: Object formation and the accretion of infrastructure," Accounting, Organizations and Society, Elsevier, vol. 47(C), pages 43-55.
    11. Murat Perit Çakır & Cengiz Acartürk & Oğuzhan Alaşehir & Canan Çilingir, 2015. "A comparative analysis of global and national university ranking systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 813-848, June.
    12. Gennaro Guida, 2018. "Italian Economics Departments’ Scientific Research Performance: Comparison between VQR and ASN Methodologies," International Journal of Business and Management, Canadian Center of Science and Education, vol. 13(9), pages 182-182, August.
    13. Gema Albort-Morant & Jörg Henseler & Antonio Leal-Millán & Gabriel Cepeda-Carrión, 2017. "Mapping the Field: A Bibliometric Analysis of Green Innovation," Sustainability, MDPI, vol. 9(6), pages 1-15, June.
    14. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Caprasecca, Alessandro, 2009. "Allocative efficiency in public research funding: Can bibliometrics help?," Research Policy, Elsevier, vol. 38(1), pages 206-215, February.
    15. Saisana, Michaela & d'Hombres, Béatrice & Saltelli, Andrea, 2011. "Rickety numbers: Volatility of university rankings and policy implications," Research Policy, Elsevier, vol. 40(1), pages 165-177, February.
    16. D. Docampo & D. Egret & L. Cram, 2015. "The effect of university mergers on the Shanghai ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 175-191, July.
    17. Bullock, Ryan & Lawler, Julia, 2015. "Community forestry research in Canada: A bibliometric perspective," Forest Policy and Economics, Elsevier, vol. 59(C), pages 47-55.
    18. Anastasios Zopiatis & Antonis L. Theocharous & Panayiotis Constanti, 2015. "‘The past is prologue to the future’: an introspective view of hospitality and tourism research," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1731-1753, February.
    19. Auranen, Otto & Nieminen, Mika, 2010. "University research funding and publication performance--An international comparison," Research Policy, Elsevier, vol. 39(6), pages 822-834, July.
    20. Ruimin Ma & Chaoqun Ni & Junping Qiu, 2008. "Scientific research competitiveness of world universities in computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(2), pages 245-260, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:72:y:2007:i:1:d:10.1007_s11192-007-1712-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.