IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v109y2016i3d10.1007_s11192-016-2147-3.html
   My bibliography  Save this article

Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators

Author

Listed:
  • Zhenbin Yan

    (University of Science and Technology of China)

  • Qiang Wu

    (University of Science and Technology of China)

  • Xingchen Li

    (University of Science and Technology of China)

Abstract

The h-index, introduced by Hirsch in 2005, was used by Schubert in 2009 to assess single publications. In 2011, Bornmann, Schier, Marx, and Daniel confirmed that the h-index is effective when assessing papers in chemistry. Quite a few Hirsch-type indices originate from the h-index. Can these Hirsch-type indices also be effectively used for assessing single publications? Will they behave the same or differently? In this study, the research objects were 26 kinds of Hirsch-type indices (including the original h-index) and three traditional methods, a total of 29 indicators. Based on the original definitions of these indicators and our new explanations of generations (i.e. mixed, pure, and non-pure generations of citations), we defined/redefined 29 paper-level metrics, calculated their values to assess publications, considered the correlations between those indices and the h-index or Wu’s w-index, and did factor analysis to contrast effectiveness. It was found that a few Hirsch-type indices (i.e. the f-index, rational h-index, real h-index, j-index, hg-index, Woeginger’s w-index, and tapered h-index) are highly correlated with the h-index but not close to Wu’s w-index, while some other indices (i.e. the a-index, h(5,2)-index, q2-index, r-index, maxprod, e-index, p-index, and weighted h-index) have relatively low correlations with the h-index but are close to Wu’s w-index. The normalized h-index and ph-ratio are obviously different from the other indices, and in most cases, their correlation coefficients with the h-index or Wu’s w-index are statistically non-significant (p > .05) or negative significant (p

Suggested Citation

  • Zhenbin Yan & Qiang Wu & Xingchen Li, 2016. "Do Hirsch-type indices behave the same in assessing single publications? An empirical study of 29 bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1815-1833, December.
    • Handle: RePEc:spr:scient:v:109:y:2016:i:3:d:10.1007_s11192-016-2147-3
    DOI: 10.1007/s11192-016-2147-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-2147-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-2147-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2008. "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 830-837, March.
    2. Kosmulski, Marek, 2013. "Family-tree of bibliometric indices," Journal of Informetrics, Elsevier, vol. 7(2), pages 313-317.
    3. Glänzel, Wolfgang & Schubert, András, 2010. "Hirsch-type characteristics of the tail of distributions. The generalised h-index," Journal of Informetrics, Elsevier, vol. 4(1), pages 118-123.
    4. John Panaretos & Chrisovaladis Malesios, 2009. "Assessing scientific research performance and impact with single indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 635-670, December.
    5. Guns, Raf & Rousseau, Ronald, 2009. "Real and rational variants of the h-index and the g-index," Journal of Informetrics, Elsevier, vol. 3(1), pages 64-71.
    6. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    7. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Markus Schaffner & Benno Torgler & Stephen Whyte, 2016. "External Influence as an Indicator of Scholarly Importance," CESifo Economic Studies, CESifo Group, vol. 62(1), pages 170-195.
    8. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    9. András Schubert, 2009. "Using the h-index for assessing single publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(3), pages 559-565, March.
    10. Roberto Todeschini, 2011. "The j-index: a new bibliometric index and multivariate comparisons between other common indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(3), pages 621-639, June.
    11. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    12. S. Alonso & F. J. Cabrerizo & E. Herrera-Viedma & F. Herrera, 2010. "hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 391-400, February.
    13. Richard S. J. Tol, 2009. "The h-index and its alternatives: An application to the 100 most prolific economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 317-324, August.
    14. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    15. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    16. Antonis Sidiropoulos & Dimitrios Katsaros & Yannis Manolopoulos, 2007. "Generalized Hirsch h-index for disclosing latent facts in citation networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(2), pages 253-280, August.
    17. L. Egghe, 2010. "On the relation between Schubert’s h-index of a single paper and its total number of received citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 115-117, July.
    18. Michael G. Banks, 2006. "An extension of the Hirsch index: Indexing scientific topics and compounds," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 161-168, October.
    19. Hu, Xiaojun & Rousseau, Ronald & Chen, Jin, 2011. "On the definition of forward and backward citation generations," Journal of Informetrics, Elsevier, vol. 5(1), pages 27-36.
    20. J Mingers, 2009. "Measuring the research contribution of management academics using the Hirsch-index," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(9), pages 1143-1153, September.
    21. Alain Molinari & Jean-Francois Molinari, 2008. "Mathematical aspects of a new criterion for ranking scientific institutions based on the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 339-356, May.
    22. Chun-Ting Zhang, 2009. "The e-Index, Complementing the h-Index for Excess Citations," PLOS ONE, Public Library of Science, vol. 4(5), pages 1-4, May.
    23. Eleni Fragkiadaki & Georgios Evangelidis, 2016. "Three novel indirect indicators for the assessment of papers and authors based on generations of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 657-694, February.
    24. Thomas R. Anderson & Robin K. S. Hankin & Peter D. Killworth, 2008. "Beyond the Durfee square: Enhancing the h-index to score total publication output," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(3), pages 577-588, September.
    25. Qiang Wu, 2010. "The w-index: A measure to assess scientific impact by focusing on widely cited papers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(3), pages 609-614, March.
    26. Frances Ruane & Richard S. J. Tol, 2008. "Rational (successive) h-indices: An application to economics in the Republic of Ireland," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(2), pages 395-405, May.
    27. Cabrerizo, F.J. & Alonso, S. & Herrera-Viedma, E. & Herrera, F., 2010. "q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core," Journal of Informetrics, Elsevier, vol. 4(1), pages 23-28.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Domingo Docampo & Jean-Jacques Bessoule, 2019. "A new approach to the analysis and evaluation of the research output of countries and institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1207-1225, May.
    2. Muhammad Raheel & Samreen Ayaz & Muhammad Tanvir Afzal, 2018. "Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1107-1127, March.
    3. Yves Fassin, 2020. "The HF-rating as a universal complement to the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 965-990, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    2. Sidiropoulos, A. & Gogoglou, A. & Katsaros, D. & Manolopoulos, Y., 2016. "Gazing at the skyline for star scientists," Journal of Informetrics, Elsevier, vol. 10(3), pages 789-813.
    3. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    4. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    5. Muhammad Raheel & Samreen Ayaz & Muhammad Tanvir Afzal, 2018. "Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1107-1127, March.
    6. Zhang, Lin & Thijs, Bart & Glänzel, Wolfgang, 2011. "The diffusion of H-related literature," Journal of Informetrics, Elsevier, vol. 5(4), pages 583-593.
    7. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    8. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    9. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    10. Qurat-ul Ain & Hira Riaz & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its citation intensity based variants in the field of mathematics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 187-211, April.
    11. Anania, Giovanni & Caruso, Annarosa, 2012. "Two New Simple Bibliometric Indexes to Better Evaluate Research in Economics," 2012 First Congress, June 4-5, 2012, Trento, Italy 124116, Italian Association of Agricultural and Applied Economics (AIEAA).
    12. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    13. Georgios Stoupas & Antonis Sidiropoulos & Antonia Gogoglou & Dimitrios Katsaros & Yannis Manolopoulos, 2018. "Rainbow ranking: an adaptable, multidimensional ranking method for publication sets," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 147-160, July.
    14. Madiha Ameer & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its qualitative and quantitative variants in Neuroscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 653-673, November.
    15. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    16. Muhammad Usman & Ghulam Mustafa & Muhammad Tanvir Afzal, 2021. "Ranking of author assessment parameters using Logistic Regression," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 335-353, January.
    17. Schreiber, M. & Malesios, C.C. & Psarakis, S., 2012. "Exploratory factor analysis for the Hirsch index, 17 h-type variants, and some traditional bibliometric indicators," Journal of Informetrics, Elsevier, vol. 6(3), pages 347-358.
    18. Eleni Fragkiadaki & Georgios Evangelidis, 2014. "Review of the indirect citations paradigm: theory and practice of the assessment of papers, authors and journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 261-288, May.
    19. Shaibu Mohammed & Emmanuel K. Nyantakyi & Anthony Morgan & Prosper Anumah & Justice Sarkodie-kyeremeh, 2021. "Use of relative extra citation counts and uncited publications to enhance the discriminatory power of the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 181-199, January.
    20. Lorna Wildgaard, 2015. "A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 873-906, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:109:y:2016:i:3:d:10.1007_s11192-016-2147-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.