IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v102y2015i1d10.1007_s11192-014-1428-y.html
   My bibliography  Save this article

Comparative analysis of some individual bibliometric indices when applied to groups of researchers

Author

Listed:
  • Ana Paula dos Santos Rubem

    (Centro de Análise de Sistemas Navais, Praça Barão de Ladário)

  • Ariane Lima Moura

    (Universidade Federal Fluminense)

  • João Carlos Correia Baptista Soares de Mello

    (Universidade Federal Fluminense)

Abstract

The h-index is a widely used bibliometric indicator for assessing individual scientists or other units of analysis. When evaluating aggregated authors, the h-index may produce rankings that are not consistent with the individual ones. The problem is claimed to affect all h-type indices; while the highly cited publications indicator, which comes from a different class, represents an alternative that is immune to such issue. The main objective of this work is to perform a comparative analysis of some bibliometric indicators originally designed to measure the overall impact of individual scientific production, when applied to the evaluation of groups, to investigate the consistency between the rankings at different levels of aggregation. For that, we use part of a previously reported citation database. The results indicate that, although the consistency at distinct aggregative levels is not formally complied by the h-index and all its variants, it is met with reasonable frequency.

Suggested Citation

  • Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
  • Handle: RePEc:spr:scient:v:102:y:2015:i:1:d:10.1007_s11192-014-1428-y
    DOI: 10.1007/s11192-014-1428-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-014-1428-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-014-1428-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    2. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    3. Woeginger, Gerhard J., 2008. "An axiomatic analysis of Egghe’s g-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 364-368.
    4. Quesada, Antonio, 2009. "Monotonicity and the Hirsch index," Journal of Informetrics, Elsevier, vol. 3(2), pages 158-160.
    5. Miroiu, Adrian, 2013. "Axiomatizing the Hirsch index: Quantity and quality disjoined," Journal of Informetrics, Elsevier, vol. 7(1), pages 10-15.
    6. John Panaretos & Chrisovaladis Malesios, 2009. "Assessing scientific research performance and impact with single indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 635-670, December.
    7. Michael Schreiber, 2010. "Revisiting the g-index: The average number of citations in the g-core," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(1), pages 169-174, January.
    8. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    9. Pablo D. Batista & Mônica G. Campiteli & Osame Kinouchi, 2006. "Is it possible to compare researchers with different scientific interests?," Scientometrics, Springer;Akadémiai Kiadó, vol. 68(1), pages 179-189, July.
    10. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    11. Ricardo Arencibia-Jorge & Ronald Rousseau, 2009. "Influence of individual researchers’ visibility on institutional impact: an example of Prathap’s approach to successive h-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 79(3), pages 507-516, June.
    12. van Eck, Nees Jan & Waltman, Ludo, 2008. "Generalizing the h- and g-indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 263-271.
    13. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    14. Antonio Quesada, 2011. "Further characterizations of the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 107-114, April.
    15. Franceschini, Fiorenzo & Maisano, Domenico A., 2010. "Analysis of the Hirsch index's operational properties," European Journal of Operational Research, Elsevier, vol. 203(2), pages 494-504, June.
    16. Vladimir G. Deineko & Gerhard J. Woeginger, 2009. "A new family of scientific impact measures: The generalized Kosmulski-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 819-826, September.
    17. Greco, Salvatore & Matarazzo, Benedetto & Slowinski, Roman, 2001. "Rough sets theory for multicriteria decision analysis," European Journal of Operational Research, Elsevier, vol. 129(1), pages 1-47, February.
    18. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    19. Thomas R. Anderson & Robin K. S. Hankin & Peter D. Killworth, 2008. "Beyond the Durfee square: Enhancing the h-index to score total publication output," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(3), pages 577-588, September.
    20. van Eck, N.J.P. & Waltman, L., 2008. "Generalizing the h- and g-indices," ERIM Report Series Research in Management ERS-2008-049-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    21. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    22. Qiang Wu, 2010. "The w-index: A measure to assess scientific impact by focusing on widely cited papers," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(3), pages 609-614, March.
    23. Tibor Braun & Wolfgang Glänzel & András Schubert, 2006. "A Hirsch-type index for journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 169-173, October.
    24. Thierry Marchant, 2009. "An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 325-342, August.
    25. S. Alonso & F. J. Cabrerizo & E. Herrera-Viedma & F. Herrera, 2010. "hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 391-400, February.
    26. Edit Csajbók & Anna Berhidi & Lívia Vasas & András Schubert, 2007. "Hirsch-index for countries based on Essential Science Indicators data," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(1), pages 91-117, October.
    27. András Schubert, 2007. "Successive h-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 70(1), pages 201-205, January.
    28. Woeginger, Gerhard J., 2008. "A symmetry axiom for scientific impact indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 298-303.
    29. Schreiber, Michael, 2008. "A modification of the h-index: The hm-index accounts for multi-authored manuscripts," Journal of Informetrics, Elsevier, vol. 2(3), pages 211-216.
    30. Cabrerizo, F.J. & Alonso, S. & Herrera-Viedma, E. & Herrera, F., 2010. "q2-Index: Quantitative and qualitative evaluation based on the number and impact of papers in the Hirsch core," Journal of Informetrics, Elsevier, vol. 4(1), pages 23-28.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    2. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    3. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    4. Deming Lin & Tianhui Gong & Wenbin Liu & Martin Meyer, 2020. "An entropy-based measure for the evolution of h index research," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2283-2298, December.
    5. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    6. Sylvain Béal & Sylvain Ferrières & Eric Rémila & Philippe Solal, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers hal-01394818, HAL.
    7. Qiang Wu & Peng Zhang, 2017. "Some indices violating the basic domination relation," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 495-500, October.
    8. L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    9. Eleni Fragkiadaki & Georgios Evangelidis, 2014. "Review of the indirect citations paradigm: theory and practice of the assessment of papers, authors and journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 261-288, May.
    10. Stefano Vannucci, 2010. "Dominance dimension: a common parametric formulation for integer-valued scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 43-48, July.
    11. Mark Levene & Trevor Fenner & Judit Bar-Ilan, 2019. "Characterisation of the $$\chi$$χ-index and the rec-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 885-896, August.
    12. Fiorenzo Franceschini & Domenico Maisano & Anna Perotti & Andrea Proto, 2010. "Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 203-217, October.
    13. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    14. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    15. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    16. Ash Mohammad Abbas, 2011. "Weighted indices for evaluating the quality of research with multiple authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 107-131, July.
    17. Fiorenzo Franceschini & Domenico Maisano, 2011. "Criticism on the hg-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 339-346, February.
    18. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    19. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    20. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:102:y:2015:i:1:d:10.1007_s11192-014-1428-y. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.