IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-01509493.html
   My bibliography  Save this paper

An axiomatic approach to bibliometric rankings and indices

Author

Listed:
  • Thierry Marchant

    (UGENT - Universiteit Gent = Ghent University)

  • Denis Bouyssou

    (LAMSADE - Laboratoire d'analyse et modélisation de systèmes pour l'aide à la décision - Université Paris Dauphine-PSL - PSL - Université Paris sciences et lettres - CNRS - Centre National de la Recherche Scientifique)

Abstract

This paper analyzes several well-known bibliometric indices using an axiomatic approach. The bibliometric indices that are studied include classic ones such as the number of highly cited papers as well as more recent ones such as the h-index and the g-index. We give conditions that characterize these indices, up to the multiplication by a positive constant. We also study the bibliometric rankings that are induced by these indices. Hence, we provide a general framework for the comparison of bibliometric rankings and indices.

Suggested Citation

  • Thierry Marchant & Denis Bouyssou, 2014. "An axiomatic approach to bibliometric rankings and indices," Post-Print hal-01509493, HAL.
    • Handle: RePEc:hal:journl:hal-01509493
    DOI: 10.1016/j.joi.2014.03.001
    Note: View the original document on HAL open archive server: https://hal.science/hal-01509493
    as

    Download full text from publisher

    File URL: https://hal.science/hal-01509493/document
    Download Restriction: no
    File URL: https://libkey.io/10.1016/j.joi.2014.03.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    2. Patrice Perny & Olivier Spanjaard & Louis-Xavier Storme, 2006. "A decision-theoretic approach to robust optimization in multivalued graphs," Annals of Operations Research, Springer, vol. 147(1), pages 317-341, October.
    3. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    4. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "High- and low-impact citation measures: Empirical applications," Journal of Informetrics, Elsevier, vol. 5(1), pages 122-145.
    5. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    6. Woeginger, Gerhard J., 2008. "An axiomatic analysis of Egghe’s g-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 364-368.
    7. Lutz Bornmann & Rüdiger Mutz & Hans‐Dieter Daniel, 2008. "Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(5), pages 830-837, March.
    8. Quesada, Antonio, 2009. "Monotonicity and the Hirsch index," Journal of Informetrics, Elsevier, vol. 3(2), pages 158-160.
    9. Kosmulski, Marek, 2013. "Family-tree of bibliometric indices," Journal of Informetrics, Elsevier, vol. 7(2), pages 313-317.
    10. Miroiu, Adrian, 2013. "Axiomatizing the Hirsch index: Quantity and quality disjoined," Journal of Informetrics, Elsevier, vol. 7(1), pages 10-15.
    11. Schreiber, Michael, 2013. "A case study of the arbitrariness of the h-index and the highly-cited-publications indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 379-387.
    12. Bornmann, Lutz & Leydesdorff, Loet, 2013. "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000," Journal of Informetrics, Elsevier, vol. 7(2), pages 286-291.
    13. Stephen Bazen & Patrick Moyes, 2012. "Elitism and stochastic dominance," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 39(1), pages 207-251, June.
    14. Marek Gągolewski & Przemysław Grzegorzewski, 2009. "A geometric approach to the construction of scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 617-634, December.
    15. Denis Bouyssou & Thierry Marchant, 2011. "Ranking scientists and departments in a consistent manner," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(9), pages 1761-1769, September.
    16. Quesada, Antonio, 2011. "Axiomatics for the Hirsch index and the Egghe index," Journal of Informetrics, Elsevier, vol. 5(3), pages 476-480.
    17. van Eck, Nees Jan & Waltman, Ludo, 2008. "Generalizing the h- and g-indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 263-271.
    18. Egghe, L., 2013. "A mathematical characterization of the Hirsch-index by means of minimal increments," Journal of Informetrics, Elsevier, vol. 7(2), pages 388-393.
    19. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    20. Antonio Quesada, 2011. "Further characterizations of the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 107-114, April.
    21. Bouyssou, D. & Marchant, T., 2010. "Consistent bibliometric rankings of authors and of journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 365-378.
    22. Bouyssou, Denis & Marchant, Thierry, 2011. "Bibliometric rankings of journals based on Impact Factors: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 5(1), pages 75-86.
    23. Franceschini, Fiorenzo & Maisano, Domenico A., 2010. "Analysis of the Hirsch index's operational properties," European Journal of Operational Research, Elsevier, vol. 203(2), pages 494-504, June.
    24. Vladimir G. Deineko & Gerhard J. Woeginger, 2009. "A new family of scientific impact measures: The generalized Kosmulski-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(3), pages 819-826, September.
    25. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    26. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    27. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h‐index," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    28. Shorrocks, Anthony F, 1983. "Ranking Income Distributions," Economica, London School of Economics and Political Science, vol. 50(197), pages 3-17, February.
    29. Waltman, Ludo & van Eck, Nees Jan & van Leeuwen, Thed N. & Visser, Martijn S. & van Raan, Anthony F.J., 2011. "Towards a new crown indicator: Some theoretical considerations," Journal of Informetrics, Elsevier, vol. 5(1), pages 37-47.
    30. Bouyssou, D. & Marchant, T., 2010. "Consistent bibliometric rankings of authors and of journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 365-378.
    31. Michael Schreiber, 2013. "Inconsistencies in the highly cited publications indicator," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(6), pages 1298-1302, June.
    32. van Eck, N.J.P. & Waltman, L., 2008. "Generalizing the h- and g-indices," ERIM Report Series Research in Management ERS-2008-049-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    33. Rousseau, Ronald, 2008. "Woeginger's axiomatisation of the h-index and its relation to the g-index, the h(2)-index and the R2-index," Journal of Informetrics, Elsevier, vol. 2(4), pages 335-340.
    34. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "The measurement of low- and high-impact in citation distributions: Technical results," Journal of Informetrics, Elsevier, vol. 5(1), pages 48-63.
    35. Waltman, L. & van Eck, N.J.P., 2009. "A Taxonomy of Bibliometric Performance Indicators Based on the Property of Consistency," ERIM Report Series Research in Management ERS-2009-014-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    36. Lutz Bornmann & Rüdiger Mutz & Hans-Dieter Daniel & Gerlind Wallon & Anna Ledin, 2009. "Are there really two types of h index variants? A validation study by using molecular life sciences data," Research Evaluation, Oxford University Press, vol. 18(3), pages 185-190, September.
    37. Yu Liu & Wei Zuo & Ying Gao & Yanhong Qiao, 2013. "Comprehensive geometrical interpretation of h-type indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 605-615, August.
    38. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    39. Waltman, L. & van Eck, N.J.P., 2009. "A simple alternative to the h-index," ERIM Report Series Research in Management ERS-2009-043-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    40. Anthony F. J. Raan, 2006. "Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 491-502, June.
    41. Rousseau, Ronald & García-Zorita, Carlos & Sanz-Casado, Elias, 2013. "The h-bubble," Journal of Informetrics, Elsevier, vol. 7(2), pages 294-300.
    42. Thierry Marchant, 2009. "An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 325-342, August.
    43. M Schreiber & C C Malesios & S Psarakis, 2011. "Categorizing h-index variants," Research Evaluation, Oxford University Press, vol. 20(5), pages 397-409, December.
    44. Hajime Eto, 2003. "Interdisciplinary information input and output of a nano-technology project," Scientometrics, Springer;Akadémiai Kiadó, vol. 58(1), pages 5-33, September.
    45. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    46. Michael Schreiber, 2013. "Inconsistencies in the highly cited publications indicator," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(6), pages 1298-1302, June.
    47. Egghe, L., 2010. "Conjugate partitions in informetrics: Lorenz curves, h-type indices, Ferrers graphs and Durfee squares in a discrete and continuous setting," Journal of Informetrics, Elsevier, vol. 4(3), pages 320-330.
    48. Woeginger, Gerhard J., 2008. "A symmetry axiom for scientific impact indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 298-303.
    49. Martin Ravallion & Adam Wagstaff, 2011. "On measuring scholarly influence by citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 321-337, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Albarrán, Pedro & Herrero, Carmen & Ruiz-Castillo, Javier & Villar, Antonio, 2017. "The Herrero-Villar approach to citation impact," Journal of Informetrics, Elsevier, vol. 11(2), pages 625-640.
    2. Qiang Wu & Peng Zhang, 2017. "Some indices violating the basic domination relation," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 495-500, October.
    3. Antonin Mac'e, 2017. "The Limits of Citation Counts," Papers 1711.02695, arXiv.org, revised Sep 2023.
    4. Sylvain Béal & Sylvain Ferrières & Eric Rémila & Phillippe Solal, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers 2016-11, CRESE.
    5. Josep Freixas & Roger Hoerl & William S. Zwicker, 2023. "Nash's bargaining problem and the scale-invariant Hirsch citation index," Papers 2309.01192, arXiv.org.
    6. L'aszl'o Csat'o & D'ora Gr'eta Petr'oczy, 2020. "Bibliometric indices as a measure of performance and competitive balance in the knockout stage of the UEFA Champions League," Papers 2005.13416, arXiv.org, revised Sep 2023.
    7. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    8. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    9. László Csató, 2018. "Characterization of an inconsistency ranking for pairwise comparison matrices," Annals of Operations Research, Springer, vol. 261(1), pages 155-165, February.
    10. Brink, René van den & Rusinowska, Agnieszka, 2021. "The degree ratio ranking method for directed graphs," European Journal of Operational Research, Elsevier, vol. 288(2), pages 563-575.
    11. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    12. Yurij L. Katchanov & Yulia V. Markova, 2017. "The “space of physics journals”: topological structure and the Journal Impact Factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 313-333, October.
    13. Mark Levene & Trevor Fenner & Judit Bar-Ilan, 2019. "Characterisation of the $$\chi$$χ-index and the rec-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 885-896, August.
    14. L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    15. Csató, László, 2019. "A characterization of the Logarithmic Least Squares Method," European Journal of Operational Research, Elsevier, vol. 276(1), pages 212-216.
    16. Mukherjee, Conan & Alam, Aftab, 2016. "On Evaluating Author's Performance by Publications: An Axiomatic Study," Working Papers 2016:14, Lund University, Department of Economics, revised 12 May 2017.
    17. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    18. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    19. Lasso de la Vega, Casilda & Volij, Oscar, 2018. "Ranking scholars: A measure representation," Journal of Informetrics, Elsevier, vol. 12(2), pages 510-517.
    20. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    21. Conan Mukherjee & Ranojoy Basu & Aftab Alam, 2020. "A measure of authorship by publications," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 41(3), pages 354-361, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ana Paula dos Santos Rubem & Ariane Lima Moura & João Carlos Correia Baptista Soares de Mello, 2015. "Comparative analysis of some individual bibliometric indices when applied to groups of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 1019-1035, January.
    2. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    3. Brandão, Luana Carneiro & Soares de Mello, João Carlos Correia Baptista, 2019. "A multi-criteria approach to the h-index," European Journal of Operational Research, Elsevier, vol. 276(1), pages 357-363.
    4. Osório, António (António Miguel), 2019. "The value and credits of n-authors publications," Working Papers 2072/376026, Universitat Rovira i Virgili, Department of Economics.
    5. Woeginger, Gerhard J., 2014. "Investigations on the step-based research indices of Chambers and Miller," Journal of Informetrics, Elsevier, vol. 8(3), pages 659-666.
    6. Chambers, Christopher P. & Miller, Alan D., 2014. "Scholarly influence," Journal of Economic Theory, Elsevier, vol. 151(C), pages 571-583.
    7. Bouyssou, Denis & Marchant, Thierry, 2011. "Bibliometric rankings of journals based on Impact Factors: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 5(1), pages 75-86.
    8. L'aszl'o Csat'o, 2019. "Journal ranking should depend on the level of aggregation," Papers 1904.06300, arXiv.org, revised Sep 2019.
    9. Gagolewski, Marek, 2013. "Scientific impact assessment cannot be fair," Journal of Informetrics, Elsevier, vol. 7(4), pages 792-802.
    10. Gagolewski, Marek & Mesiar, Radko, 2012. "Aggregating different paper quality measures with a generalized h-index," Journal of Informetrics, Elsevier, vol. 6(4), pages 566-579.
    11. Csató, László, 2019. "Journal ranking should depend on the level of aggregation," Journal of Informetrics, Elsevier, vol. 13(4).
    12. Stefano Vannucci, 2010. "Dominance dimension: a common parametric formulation for integer-valued scientific impact indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 43-48, July.
    13. Sylvain Béal & Eric Rémila & Philippe Solal & Sylvain Ferrières, 2016. "An axiomatization of the iterated h-index and applications to sport rankings," Working Papers hal-01394818, HAL.
    14. Javier Ruiz-Castillo, 2012. "The evaluation of citation distributions," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
    15. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    16. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    17. Marcin Kozak & Lutz Bornmann, 2012. "A New Family of Cumulative Indexes for Measuring Scientific Performance," PLOS ONE, Public Library of Science, vol. 7(10), pages 1-4, October.
    18. Bornmann, Lutz & Mutz, Rüdiger & Hug, Sven E. & Daniel, Hans-Dieter, 2011. "A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants," Journal of Informetrics, Elsevier, vol. 5(3), pages 346-359.
    19. Josep Freixas & Roger Hoerl & William S. Zwicker, 2023. "Nash's bargaining problem and the scale-invariant Hirsch citation index," Papers 2309.01192, arXiv.org.
    20. Madiha Ameer & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its qualitative and quantitative variants in Neuroscience," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 653-673, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-01509493. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.