IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0316913.html
   My bibliography  Save this article

Inequality in measuring scholarly success: Variation in the h-index within and between disciplines

Author

Listed:
  • Ryan Light
  • Aaron Gullickson
  • Jill Ann Harrison

Abstract

Scholars and university administrators have a vested interest in building equitable valuation systems of academic work for both practical (e.g., resource distribution) and more lofty purposes (e.g., what constitutes “good” research). Well-established inequalities in science pose a difficult challenge to those interested in constructing a parsimonious and fair method for valuation as stratification occurs within academic disciplines, but also between them. The h-index, a popular research metric, has been formally used as one such method of valuation. In this article, we use the case of the h-index to examine how the distribution of research metrics reveal within and between discipline inequalities. Using bibliometric data from 1960-2019 on over 50,000 high performing scientists—the top 2% most frequently cited authors—across 174 disciplines, we construct random effects within-between models predicting the h-index. Results suggest significant within-discipline variation in several forms, specifically sole-authorship and female penalties. Results also show that a sole authorship penalty plays a significant role in well-known between-discipline variation. Field-specific models emphasize the “apples-to-oranges,” or incommensurable, property of cross-discipline comparison with significant heterogeneity in sole-authorship and female penalties within fields. In conclusion, we recommend continued caution when using the h-index or similar metrics for valuation purposes and the prioritization of substantive valuations from disciplinary experts.

Suggested Citation

  • Ryan Light & Aaron Gullickson & Jill Ann Harrison, 2025. "Inequality in measuring scholarly success: Variation in the h-index within and between disciplines," PLOS ONE, Public Library of Science, vol. 20(1), pages 1-16, January.
    • Handle: RePEc:plo:pone00:0316913
    DOI: 10.1371/journal.pone.0316913
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0316913
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0316913&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0316913?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mohsen Jadidi & Fariba Karimi & Haiko Lietz & Claudia Wagner, 2018. "Gender Disparities In Science? Dropout, Productivity, Collaborations And Success Of Male And Female Computer Scientists," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 21(03n04), pages 1-23, May.
    2. Henry H. Bi, 2023. "Correction to: Four problems of the h-index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2693-2699, May.
    3. Björn Hammarfelt & Gaby Haddow, 2018. "Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 924-935, July.
    4. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    5. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    6. Weeden, Kim & Thèbaud, Sarah & Gelbgiser, Dafna, 2017. "Degrees of Difference: Gender Segregation of U.S. Doctorates by Field and Program Prestige," OSF Preprints 9k6tv, Center for Open Science.
    7. John P A Ioannidis & Kevin W Boyack & Jeroen Baas, 2020. "Updated science-wide author databases of standardized citation indicators," PLOS Biology, Public Library of Science, vol. 18(10), pages 1-3, October.
    8. Nikki Forrester, 2021. "Mental health of graduate students sorely overlooked," Nature, Nature, vol. 595(7865), pages 135-137, July.
    9. Vivek Kumar Singh & Prashasti Singh & Mousumi Karmakar & Jacqueline Leta & Philipp Mayr, 2021. "The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5113-5142, June.
    10. Margaret K. Merga & Sayidi Mat Roni & Shannon Mason, 2020. "Should Google Scholar be used for benchmarking against the professoriate in education?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2505-2522, December.
    11. repec:osf:osfxxx:9k6tv_v1 is not listed on IDEAS
    12. Henry H. Bi, 2023. "Four problems of the h-index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2677-2691, May.
    13. Henry H. Bi, 2023. "Correction to: Four problems of the h‑index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2701-2701, May.
    14. repec:plo:pbio00:3000384 is not listed on IDEAS
    15. Siler, Kyle & Larivière, Vincent, 2022. "Who games metrics and rankings? Institutional niches and journal impact factor inflation," Research Policy, Elsevier, vol. 51(10).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Marek Kwiek & Wojciech Roszka, 2022. "Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3543-3575, June.
    3. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    4. Judith Nyulas & Ștefan Dezsi & Adrian-Florin Niță & Zsolt Magyari-Sáska & Marie-Luise Frey & Alpár Horváth, 2025. "Twenty-Five Years of Scientific Production on Geoparks from the Perspective of Bibliometric Analysis Using PRISMA," Sustainability, MDPI, vol. 17(5), pages 1-30, March.
    5. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.
    6. Dušan Nikolić & Dragan Ivanović & Lidija Ivanović, 2024. "An open-source tool for merging data from multiple citation databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4573-4595, July.
    7. Philippe Aghion & Celine Antonin & Luc Paluskiewicz & David Stromberg & Xueping Sun & Raphael Wargon, 2023. "Does Chinese research hinge on US co-authors? Evidence from the China Initiative," POID Working Papers 073, Centre for Economic Performance, LSE.
    8. Jordi Ardanuy & Llorenç Arguimbau & Ángel Borrego, 2022. "Social sciences and humanities research funded under the European Union Sixth Framework Programme (2002–2006): a long-term assessment of projects, acknowledgements and publications," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-13, December.
    9. Thamyres T. Choji & Manuel J. Cobo & Jose A. Moral-Munoz, 2024. "Is the scientific impact of the LIS themes gender-biased? A bibliometric analysis of the evolution, scientific impact, and relative contribution by gender from 2007 to 2022," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6023-6047, October.
    10. Gabriel Alves Vieira & Jacqueline Leta, 2024. "biblioverlap: an R package for document matching across bibliographic datasets," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4513-4527, July.
    11. Haochen Qian & Fan Zhang & Bing Qiu, 2023. "Deciphering the Evolution, Frontier, and Knowledge Clustering in Sustainable City Planning: A 60-Year Interdisciplinary Review," Sustainability, MDPI, vol. 15(24), pages 1-27, December.
    12. Katerina Guba, 2024. "Why do sociologists on academic periphery willingly support bibliometric indicators?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 497-518, January.
    13. Vivek Kumar Singh & Prashasti Singh & Ashraf Uddin & Parveen Arora & Sujit Bhattacharya, 2022. "Exploring the relationship between journals indexed from a country and its research output: an empirical investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 2933-2966, June.
    14. Raf Vanderstraeten & Frédéric Vandermoere, 2021. "Inequalities in the growth of Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8635-8651, October.
    15. Hamid R. Jamali & Alireza Abbasi, 2023. "Gender gaps in Australian research publishing, citation and co-authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2879-2893, May.
    16. Michael Gusenbauer, 2022. "Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2683-2745, May.
    17. Toluwase Asubiaro & Sodiq Onaolapo & David Mills, 2024. "Regional disparities in Web of Science and Scopus journal coverage," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1469-1491, March.
    18. Claudia N. Gonzalez Brambila & Renata Herrerias, 2024. "Assessing the impact of collaborative authorship in Business Economics in Latin America," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4623-4660, July.
    19. Kayvan Kousha & Mike Thelwall, 2024. "Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(3), pages 215-244, March.
    20. Shir Aviv-Reuven & Ariel Rosenfeld, 2023. "A logical set theory approach to journal subject classification analysis: intra-system irregularities and inter-system discrepancies in Web of Science and Scopus," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 157-175, January.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0316913. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.