IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v130y2025i1d10.1007_s11192-024-05178-x.html
   My bibliography  Save this article

Assessment of subject-normalized comprehensiveness of research-intensive universities

Author

Listed:
  • Saulo Mendes

    (University of Geneva
    Shanghai Jiao Tong University)

Abstract

Evaluation of university league tables is a controversial topic in higher education studies and metrics of scientific evaluation. Since its appearance on an international scale two decades ago, significant criticism of their methodologies has been accumulating without being addressed. One important bias regards the unequal distribution of research output and impact across different subjects, which in turn favors institutions that have the strongest performance in medical and physical sciences. The university league tables are revisited by normalizing each field to create a uniform distribution of value. Then, the overall performance of an institution is interpreted as the probability of having a high score in any given academic field while not inducing a zero-sum game. The present assessment shows that the main difference between hundreds of leading research universities is whether their coverage of all areas of human knowledge is comprehensive or specialized, whereas their mean performance per subject is nearly indistinguishable.

Suggested Citation

  • Saulo Mendes, 2025. "Assessment of subject-normalized comprehensiveness of research-intensive universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(1), pages 109-131, January.
    • Handle: RePEc:spr:scient:v:130:y:2025:i:1:d:10.1007_s11192-024-05178-x
    DOI: 10.1007/s11192-024-05178-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05178-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05178-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Valero, Anna & Van Reenen, John, 2019. "The economic impact of universities: Evidence from across the globe," Economics of Education Review, Elsevier, vol. 68(C), pages 53-67.
    2. Can Dai & Quan Chen & Tao Wan & Fan Liu & Yanbing Gong & Qingfeng Wang, 2021. "Literary runaway: Increasingly more references cited per academic research article from 1980 to 2019," PLOS ONE, Public Library of Science, vol. 16(8), pages 1-13, August.
    3. Mannella, Riccardo & Rossi, Paolo, 2013. "On the time dependence of the h-index," Journal of Informetrics, Elsevier, vol. 7(1), pages 176-182.
    4. Davide Cantoni & Noam Yuchtman, 2014. "Medieval Universities, Legal Institutions, and the Commercial Revolution," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(2), pages 823-887.
    5. Mohammad Hossein Biglu, 2008. "The influence of references per paper in the SCI to Impact Factors and the Matthew Effect," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(3), pages 453-470, March.
    6. Benjamin M. Althouse & Jevin D. West & Carl T. Bergstrom & Theodore Bergstrom, 2009. "Differences in impact factor across fields and over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 27-34, January.
    7. Philip G. Altbach & Jamil Salmi, 2011. "The Road to Academic Excellence : The Making of World-Class Research Universities [El camino hacia la excelencia académica : la constitución de universidades de investigación de rango mundial]," World Bank Publications - Books, The World Bank Group, number 2357, August.
    8. Luc Anselin & Attila Varga & Zoltan Acs, 2008. "Local Geographic Spillovers Between University Research and High Technology Innovations," Chapters, in: Entrepreneurship, Growth and Public Policy, chapter 9, pages 95-121, Edward Elgar Publishing.
    9. Sangwal, K., 2011. "On the growth of citations of publication output of individual authors," Journal of Informetrics, Elsevier, vol. 5(4), pages 554-564.
    10. Meng-Chen Zhang & Bo-Wei Zhu & Chao-Meng Huang & Gwo-Hshiung Tzeng, 2021. "Systematic Evaluation Model for Developing Sustainable World-Class Universities: An East Asian Perspective," Mathematics, MDPI, vol. 9(8), pages 1-20, April.
    11. Domingo Docampo, 2013. "Reproducibility of the Shanghai academic ranking of world universities results," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 567-587, February.
    12. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    13. Kuang-hua Chen & Pei-yu Liao, 2012. "A comparative study on world university rankings: a bibliometric survey," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(1), pages 89-103, July.
    14. Leo Egghe, 2007. "Dynamic h‐index: The Hirsch index in function of time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(3), pages 452-454, February.
    15. Finardi, Ugo, 2014. "On the time evolution of received citations, in different scientific fields: An empirical study," Journal of Informetrics, Elsevier, vol. 8(1), pages 13-24.
    16. Kurt Larsen & John P. Martin & Rosemary Morris, 2002. "Trade in Educational Services: Trends and Emerging Issues," The World Economy, Wiley Blackwell, vol. 25(6), pages 849-868, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Valero, Anna & Van Reenen, John, 2019. "The economic impact of universities: Evidence from across the globe," Economics of Education Review, Elsevier, vol. 68(C), pages 53-67.
    2. Sivropoulos-Valero, Anna Alexandra, 2021. "Education and economic growth," LSE Research Online Documents on Economics 114434, London School of Economics and Political Science, LSE Library.
    3. Nabil Amara & Réjean Landry & Norrin Halilem, 2015. "What can university administrators do to increase the publication and citation scores of their faculty members?," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 489-530, May.
    4. Anna Valero, 2021. "Education and economic growth," POID Working Papers 006, Centre for Economic Performance, LSE.
    5. Alexandra López Cermeño, 2017. "Universities, spillovers and the resilience of inequality in the human-capital century," Working Papers 17016, Economic History Society.
    6. Samia FERHAT, 2022. "The impact of university openings on labor market outcomes," THEMA Working Papers 2022-18, THEMA (THéorie Economique, Modélisation et Applications), Université de Cergy-Pontoise.
    7. Anna Valero, 2021. "Education and economic growth," CEP Discussion Papers dp1764, Centre for Economic Performance, LSE.
    8. Yanagiura, Takeshi & Tateishi, Shinji, 2024. "Local economic impact of small, non-research private universities: evidence from Japan," Economics of Education Review, Elsevier, vol. 102(C).
    9. Tobias Schlegel & Curdin Pfister & Dietmar Harhoff & Uschi Backes-Gellner, 2022. "Innovation effects of universities of applied sciences: an assessment of regional heterogeneity," The Journal of Technology Transfer, Springer, vol. 47(1), pages 63-118, February.
    10. Bautista, M. A. & González, F. & Martínez, L. R. & Muñoz, P. & Prem, M., 2020. "Chile’s Missing Students: Dictatorship, Higher Education and Social Mobility," Documentos de Trabajo 18163, Universidad del Rosario.
    11. Joan Crespo & Jesús Peiró-Palomino & Emili Tortosa-Ausina, 2020. "Does university performance have an economic payoff for their home regions? Evidence for the Spanish provinces," Working Papers 2020/20, Economics Department, Universitat Jaume I, Castellón (Spain).
    12. Oussama Ben Atta, 2022. "University proximity at teenage years and educational attainment," French Stata Users' Group Meetings 2022 02, Stata Users Group.
    13. Milojević, Staša & Radicchi, Filippo & Bar-Ilan, Judit, 2017. "Citation success index − An intuitive pair-wise journal comparison metric," Journal of Informetrics, Elsevier, vol. 11(1), pages 223-231.
    14. Robert Calvert Jump & Adam Scavette, 2024. "Do Research Universities Recession Proof Their Regions? Evidence from State Flagship College Towns," Working Paper 24-05, Federal Reserve Bank of Richmond.
    15. Sivropoulos-Valero, Anna Valero, 2021. "Education and management practices," LSE Research Online Documents on Economics 114436, London School of Economics and Political Science, LSE Library.
    16. Pfister, Curdin & Koomen, Miriam & Harhoff, Dietmar & Backes-Gellner, Uschi, 2021. "Regional innovation effects of applied research institutions," Research Policy, Elsevier, vol. 50(4).
    17. George Abuchi Agwu & Oussama Ben Atta, 2021. "University proximity at teenage years and educational attainment," Working papers of Transitions Energétiques et Environnementales (TREE) hal-03492963, HAL.
    18. Andy Feng & Anna Valero, 2020. "Skill-Biased Management: Evidence from Manufacturing Firms," The Economic Journal, Royal Economic Society, vol. 130(628), pages 1057-1080.
    19. Loet Leydesdorff & Ping Zhou & Lutz Bornmann, 2013. "How can journal impact factors be normalized across fields of science? An assessment in terms of percentile ranks and fractional counts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 96-107, January.
    20. Antonoyiannakis, Manolis, 2018. "Impact Factors and the Central Limit Theorem: Why citation averages are scale dependent," Journal of Informetrics, Elsevier, vol. 12(4), pages 1072-1088.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:130:y:2025:i:1:d:10.1007_s11192-024-05178-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.