IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0316913.html
   My bibliography  Save this article

Inequality in measuring scholarly success: Variation in the h-index within and between disciplines

Author

Listed:
  • Ryan Light
  • Aaron Gullickson
  • Jill Ann Harrison

Abstract

Scholars and university administrators have a vested interest in building equitable valuation systems of academic work for both practical (e.g., resource distribution) and more lofty purposes (e.g., what constitutes “good” research). Well-established inequalities in science pose a difficult challenge to those interested in constructing a parsimonious and fair method for valuation as stratification occurs within academic disciplines, but also between them. The h-index, a popular research metric, has been formally used as one such method of valuation. In this article, we use the case of the h-index to examine how the distribution of research metrics reveal within and between discipline inequalities. Using bibliometric data from 1960-2019 on over 50,000 high performing scientists—the top 2% most frequently cited authors—across 174 disciplines, we construct random effects within-between models predicting the h-index. Results suggest significant within-discipline variation in several forms, specifically sole-authorship and female penalties. Results also show that a sole authorship penalty plays a significant role in well-known between-discipline variation. Field-specific models emphasize the “apples-to-oranges,” or incommensurable, property of cross-discipline comparison with significant heterogeneity in sole-authorship and female penalties within fields. In conclusion, we recommend continued caution when using the h-index or similar metrics for valuation purposes and the prioritization of substantive valuations from disciplinary experts.

Suggested Citation

  • Ryan Light & Aaron Gullickson & Jill Ann Harrison, 2025. "Inequality in measuring scholarly success: Variation in the h-index within and between disciplines," PLOS ONE, Public Library of Science, vol. 20(1), pages 1-16, January.
    • Handle: RePEc:plo:pone00:0316913
    DOI: 10.1371/journal.pone.0316913
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0316913
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0316913&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0316913?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    2. repec:osf:osfxxx:9k6tv_v1 is not listed on IDEAS
    3. Henry H. Bi, 2023. "Correction to: Four problems of the h-index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2693-2699, May.
    4. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    5. Henry H. Bi, 2023. "Correction to: Four problems of the h‑index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2701-2701, May.
    6. Margaret K. Merga & Sayidi Mat Roni & Shannon Mason, 2020. "Should Google Scholar be used for benchmarking against the professoriate in education?," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2505-2522, December.
    7. John P A Ioannidis & Kevin W Boyack & Jeroen Baas, 2020. "Updated science-wide author databases of standardized citation indicators," PLOS Biology, Public Library of Science, vol. 18(10), pages 1-3, October.
    8. Nikki Forrester, 2021. "Mental health of graduate students sorely overlooked," Nature, Nature, vol. 595(7865), pages 135-137, July.
    9. Vivek Kumar Singh & Prashasti Singh & Mousumi Karmakar & Jacqueline Leta & Philipp Mayr, 2021. "The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5113-5142, June.
    10. Henry H. Bi, 2023. "Four problems of the h-index for assessing the research productivity and impact of individual authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2677-2691, May.
    11. Björn Hammarfelt & Gaby Haddow, 2018. "Conflicting measures and values: How humanities scholars in Australia and Sweden use and react to bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 924-935, July.
    12. Siler, Kyle & Larivière, Vincent, 2022. "Who games metrics and rankings? Institutional niches and journal impact factor inflation," Research Policy, Elsevier, vol. 51(10).
    13. Weeden, Kim & Thèbaud, Sarah & Gelbgiser, Dafna, 2017. "Degrees of Difference: Gender Segregation of U.S. Doctorates by Field and Program Prestige," OSF Preprints 9k6tv, Center for Open Science.
    14. repec:plo:pbio00:3000384 is not listed on IDEAS
    15. Mohsen Jadidi & Fariba Karimi & Haiko Lietz & Claudia Wagner, 2018. "Gender Disparities In Science? Dropout, Productivity, Collaborations And Success Of Male And Female Computer Scientists," Advances in Complex Systems (ACS), World Scientific Publishing Co. Pte. Ltd., vol. 21(03n04), pages 1-23, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yang, Alex Jie & Wu, Linwei & Zhang, Qi & Wang, Hao & Deng, Sanhong, 2023. "The k-step h-index in citation networks at the paper, author, and institution levels," Journal of Informetrics, Elsevier, vol. 17(4).
    2. Marek Kwiek & Wojciech Roszka, 2022. "Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3543-3575, June.
    3. Pantea Kamrani & Isabelle Dorsch & Wolfgang G. Stock, 2021. "Do researchers know what the h-index is? And how do they estimate its importance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5489-5508, July.
    4. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.
    5. Dušan Nikolić & Dragan Ivanović & Lidija Ivanović, 2024. "An open-source tool for merging data from multiple citation databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4573-4595, July.
    6. Jordi Ardanuy & Llorenç Arguimbau & Ángel Borrego, 2022. "Social sciences and humanities research funded under the European Union Sixth Framework Programme (2002–2006): a long-term assessment of projects, acknowledgements and publications," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-13, December.
    7. Katerina Guba, 2024. "Why do sociologists on academic periphery willingly support bibliometric indicators?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 497-518, January.
    8. Raf Vanderstraeten & Frédéric Vandermoere, 2021. "Inequalities in the growth of Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8635-8651, October.
    9. Hamid R. Jamali & Alireza Abbasi, 2023. "Gender gaps in Australian research publishing, citation and co-authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2879-2893, May.
    10. Toluwase Asubiaro & Sodiq Onaolapo & David Mills, 2024. "Regional disparities in Web of Science and Scopus journal coverage," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1469-1491, March.
    11. Abdelkebir Sahid & Yassine Maleh & Karim Ouazzane, 2025. "Changing landscape of fake news research on social media: a bibliometric analysis," Quality & Quantity: International Journal of Methodology, Springer, vol. 59(2), pages 901-954, April.
    12. Biru Desta Kassaye & Yitbarek Takele Bayiley & Zerihun Kinde Alemu, 2025. "Returnee migrant entrepreneurship: a bibliometric analysis (1993–2024)," Journal of Global Entrepreneurship Research, Springer;UNESCO Chair in Entrepreneurship, vol. 15(1), pages 1-23, December.
    13. Lina Mosquera Chaverra & Diego Paredes Cuervo & Ana López Gutiérrez & Carlos A. Arias & Pedro N. Carvalho, 2024. "Phytoremediation of Mercury Contamination: Bibliometric Analysis," Sustainability, MDPI, vol. 16(21), pages 1-28, October.
    14. Kumar, Satish & Sahoo, Saumyaranjan & Lim, Weng Marc & Kraus, Sascha & Bamel, Umesh, 2022. "Fuzzy-set qualitative comparative analysis (fsQCA) in business and management research: A contemporary overview," Technological Forecasting and Social Change, Elsevier, vol. 178(C).
    15. Adriana Ana Maria Davidescu & Margareta-Stela Florescu & Liviu Cosmin Mosora & Mihaela Hrisanta Mosora & Eduard Mihai Manta, 2022. "A Bibliometric Analysis of Research Publications of the Bucharest University of Economic Studies in Time of Pandemics: Implications for Teachers’ Professional Publishing Activity," IJERPH, MDPI, vol. 19(14), pages 1-36, July.
    16. Weishu Liu & Rong Ni & Guangyuan Hu, 2024. "Web of Science Core Collection’s coverage expansion: the forgotten Arts & Humanities Citation Index?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(2), pages 933-955, February.
    17. Congying Fang & Riken Homma & Tianfu Qiu, 2024. "A Bibliometrics Analysis Related to the Built Environment and Walking," Sustainability, MDPI, vol. 16(7), pages 1-17, March.
    18. Judith Nyulas & Ștefan Dezsi & Adrian-Florin Niță & Zsolt Magyari-Sáska & Marie-Luise Frey & Alpár Horváth, 2025. "Twenty-Five Years of Scientific Production on Geoparks from the Perspective of Bibliometric Analysis Using PRISMA," Sustainability, MDPI, vol. 17(5), pages 1-30, March.
    19. Philippe Aghion & Celine Antonin & Luc Paluskiewicz & David Stromberg & Raphael Wargon & Karolina Westin, 2023. "Does Chinese research hinge on US co-authors? Evidence from the China initiative," CEP Discussion Papers dp1936, Centre for Economic Performance, LSE.
    20. Thamyres T. Choji & Manuel J. Cobo & Jose A. Moral-Munoz, 2024. "Is the scientific impact of the LIS themes gender-biased? A bibliometric analysis of the evolution, scientific impact, and relative contribution by gender from 2007 to 2022," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(10), pages 6023-6047, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0316913. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.