IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v8y2014i3p449-477.html
   My bibliography  Save this article

An axiomatic approach to bibliometric rankings and indices

Author

Listed:
  • Bouyssou, Denis
  • Marchant, Thierry

Abstract

This paper analyzes several well-known bibliometric indices using an axiomatic approach. We concentrate on indices aiming at capturing the global impact of a scientific output and do not investigate indices aiming at capturing an average impact. Hence, the indices that we study are designed to evaluate authors or groups of authors but not journals. The bibliometric indices that are studied include classic ones such as the number of highly cited papers as well as more recent ones such as the h-index and the g-index. We give conditions that characterize these indices, up to the multiplication by a positive constant. We also study the bibliometric rankings that are induced by these indices. Hence, we provide a general framework for the comparison of bibliometric rankings and indices.

Suggested Citation

  • Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
  • Handle: RePEc:eee:infome:v:8:y:2014:i:3:p:449-477
    DOI: 10.1016/j.joi.2014.03.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157714000297
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    2. Patrice Perny & Olivier Spanjaard & Louis-Xavier Storme, 2006. "A decision-theoretic approach to robust optimization in multivalued graphs," Annals of Operations Research, Springer, vol. 147(1), pages 317-341, October.
    3. Franceschini, Fiorenzo & Maisano, Domenico A., 2010. "Analysis of the Hirsch index's operational properties," European Journal of Operational Research, Elsevier, vol. 203(2), pages 494-504, June.
    4. Vladimir G. Deineko & Gerhard J. Woeginger, 2009. "A new family of scientific impact measures: The generalized Kosmulski-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 819-826, September.
    5. Bouyssou, Denis & Marchant, Thierry, 2011. "Bibliometric rankings of journals based on Impact Factors: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 5(1), pages 75-86.
    6. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    7. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "High- and low-impact citation measures: Empirical applications," Journal of Informetrics, Elsevier, vol. 5(1), pages 122-145.
    8. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    9. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    10. Woeginger, Gerhard J., 2008. "An axiomatic analysis of Egghe’s g-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 364-368.
    11. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h‐index," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    12. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2008. "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 830-837, March.
    13. Quesada, Antonio, 2009. "Monotonicity and the Hirsch index," Journal of Informetrics, Elsevier, vol. 3(2), pages 158-160.
    14. Denis Bouyssou & Thierry Marchant, 2011. "Ranking scientists and departments in a consistent manner," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(9), pages 1761-1769, September.
    15. Shorrocks, Anthony F, 1983. "Ranking Income Distributions," Economica, London School of Economics and Political Science, vol. 50(197), pages 3-17, February.
    16. Kosmulski, Marek, 2013. "Family-tree of bibliometric indices," Journal of Informetrics, Elsevier, vol. 7(2), pages 313-317.
    17. Miroiu, Adrian, 2013. "Axiomatizing the Hirsch index: Quantity and quality disjoined," Journal of Informetrics, Elsevier, vol. 7(1), pages 10-15.
    18. Waltman, Ludo & van Eck, Nees Jan & van Leeuwen, Thed N. & Visser, Martijn S. & van Raan, Anthony F.J., 2011. "Towards a new crown indicator: Some theoretical considerations," Journal of Informetrics, Elsevier, vol. 5(1), pages 37-47.
    19. Schreiber, Michael, 2013. "A case study of the arbitrariness of the h-index and the highly-cited-publications indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 379-387.
    20. Bouyssou, D. & Marchant, T., 2010. "Consistent bibliometric rankings of authors and of journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 365-378.
    21. van Eck, N.J.P. & Waltman, L., 2008. "Generalizing the h- and g-indices," ERIM Report Series Research in Management ERS-2008-049-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    22. Bornmann, Lutz & Leydesdorff, Loet, 2013. "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000," Journal of Informetrics, Elsevier, vol. 7(2), pages 286-291.
    23. Rousseau, Ronald, 2008. "Woeginger's axiomatisation of the h-index and its relation to the g-index, the h(2)-index and the R2-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 335-340.
    24. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "The measurement of low- and high-impact in citation distributions: Technical results," Journal of Informetrics, Elsevier, vol. 5(1), pages 48-63.
    25. Lutz Bornmann & Rüdiger Mutz & Hans-Dieter Daniel & Gerlind Wallon & Anna Ledin, 2009. "Are there really two types of h index variants? A validation study by using molecular life sciences data," Research Evaluation, Oxford University Press, vol. 18(3), pages 185-190, September.
    26. Stephen Bazen & Patrick Moyes, 2012. "Elitism and stochastic dominance," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 39(1), pages 207-251, June.
    27. Yu Liu & Wei Zuo & Ying Gao & Yanhong Qiao, 2013. "Comprehensive geometrical interpretation of h-type indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 605-615, August.
    28. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    29. Marek Gągolewski & Przemysław Grzegorzewski, 2009. "A geometric approach to the construction of scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 617-634, December.
    30. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    31. Waltman, L. & van Eck, N.J.P., 2009. "A simple alternative to the h-index," ERIM Report Series Research in Management ERS-2009-043-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    32. Rousseau, Ronald & García-Zorita, Carlos & Sanz-Casado, Elias, 2013. "The h-bubble," Journal of Informetrics, Elsevier, vol. 7(2), pages 294-300.
    33. M Schreiber & C C Malesios & S Psarakis, 2011. "Categorizing h-index variants," Research Evaluation, Oxford University Press, vol. 20(5), pages 397-409, December.
    34. van Eck, Nees Jan & Waltman, Ludo, 2008. "Generalizing the h- and g-indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 263-271.
    35. Egghe, L., 2013. "A mathematical characterization of the Hirsch-index by means of minimal increments," Journal of Informetrics, Elsevier, vol. 7(2), pages 388-393.
    36. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    37. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    38. Antonio Quesada, 2011. "Further characterizations of the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 107-114, April.
    39. Woeginger, Gerhard J., 2008. "A symmetry axiom for scientific impact indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 298-303.
    40. Martin Ravallion & Adam Wagstaff, 2011. "On measuring scholarly influence by citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 321-337, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Albarrán, Pedro & Herrero, Carmen & Ruiz-Castillo, Javier & Villar, Antonio, 2017. "The Herrero-Villar approach to citation impact," Journal of Informetrics, Elsevier, vol. 11(2), pages 625-640.
    2. L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    3. Csató, László, 2019. "A characterization of the Logarithmic Least Squares Method," European Journal of Operational Research, Elsevier, vol. 276(1), pages 212-216.
    4. Qiang Wu & Peng Zhang, 2017. "Some indices violating the basic domination relation," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 495-500, October.
    5. Mukherjee, Conan & Alam, Aftab, 2016. "On Evaluating Author's Performance by Publications: An Axiomatic Study," Working Papers 2016:14, Lund University, Department of Economics, revised 12 May 2017.
    6. Sylvain Béal & Sylvain Ferrières & Eric Rémila & Phillippe Solal, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers 2016-11, CRESE.
    7. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    8. L'aszl'o Csat'o & D'ora Gr'eta Petr'oczy, 2020. "Bibliometric indices as a measure of long-term competitive balance in knockout tournaments," Papers 2005.13416, arXiv.org, revised Sep 2020.
    9. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    10. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    11. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    12. László Csató, 2018. "Characterization of an inconsistency ranking for pairwise comparison matrices," Annals of Operations Research, Springer, vol. 261(1), pages 155-165, February.
    13. Lasso de la Vega, Casilda & Volij, Oscar, 2018. "Ranking scholars: A measure representation," Journal of Informetrics, Elsevier, vol. 12(2), pages 510-517.
    14. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    15. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    16. Brink, René van den & Rusinowska, Agnieszka, 2021. "The degree ratio ranking method for directed graphs," European Journal of Operational Research, Elsevier, vol. 288(2), pages 563-575.
    17. Conan Mukherjee & Ranojoy Basu & Aftab Alam, 2020. "A measure of authorship by publications," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(3), pages 354-361, April.
    18. Yurij L. Katchanov & Yulia V. Markova, 2017. "The “space of physics journals”: topological structure and the Journal Impact Factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 313-333, October.
    19. Mark Levene & Trevor Fenner & Judit Bar-Ilan, 2019. "Characterisation of the $$\chi$$χ-index and the rec-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 885-896, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    2. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    3. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    4. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    5. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    6. Sylvain Béal & Sylvain Ferrières & Eric Rémila & Philippe Solal, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers hal-01394818, HAL.
    7. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    8. Stefano Vannucci, 2010. "Dominance dimension: a common parametric formulation for integer-valued scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 43-48, July.
    9. Javier Ruiz-Castillo, 2012. "The evaluation of citation distributions," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
    10. L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    11. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    12. Madiha Ameer & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its qualitative and quantitative variants in Neuroscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 653-673, November.
    13. Bouyssou, Denis & Marchant, Thierry, 2011. "Bibliometric rankings of journals based on Impact Factors: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 5(1), pages 75-86.
    14. Antonio Abatemarco & Roberto Dell’Anno, 2013. "Certainty equivalent citation: generalized classes of citation indexes," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 263-271, January.
    15. Yves Fassin, 2020. "The HF-rating as a universal complement to the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 965-990, November.
    16. Dilruba Mahbuba & Ronald Rousseau, 2013. "Year-based h-type indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 785-797, September.
    17. Gagolewski, Marek, 2013. "Scientific impact assessment cannot be fair," Journal of Informetrics, Elsevier, vol. 7(4), pages 792-802.
    18. Gagolewski, Marek & Mesiar, Radko, 2012. "Aggregating different paper quality measures with a generalized h-index," Journal of Informetrics, Elsevier, vol. 6(4), pages 566-579.
    19. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    20. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:8:y:2014:i:3:p:449-477. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Nithya Sathishkumar). General contact details of provider: http://www.elsevier.com/locate/joi .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.