IDEAS home Printed from https://ideas.repec.org/p/cte/werepe/we095735.html
   My bibliography  Save this paper

The measurement of low- and high-impact in citation distributions : technical results

Author

Listed:
  • Albarrán, Pedro
  • Ortuño, Ignacio
  • Ruiz-Castillo, Javier

Abstract

This paper introduces a novel methodology for comparing the citation distributions of research units working in the same homogeneous field. Given a critical citation level (CCL), we suggest using two real valued indicators to describe the shape of any distribution: a high-impact and a low-impact measure defined over the set of articles with citations above or below the CCL. The key to this methodology is the identification of a citation distribution with an income distribution. Once this step is taken, it is easy to realize that the measurement of low-impact coincides with the measurement of economic poverty. In turn, it is equally natural to identify the measurement of high-impact with the measurement of a certain notion of economic affluence. On the other hand, it is seen that the ranking of citation distributions according to a family of low-impact measures is essentially characterized by a number of desirable axioms. Appropriately redefined, these same axioms lead to the selection of an equally convenient class of decomposable high-impact measures. These two families are shown to satisfy other interesting properties that make them potentially useful in empirical applications, including the comparison of research units working in different fields.

Suggested Citation

  • Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2009. "The measurement of low- and high-impact in citation distributions : technical results," UC3M Working papers. Economics we095735, Universidad Carlos III de Madrid. Departamento de Economía.
    • Handle: RePEc:cte:werepe:we095735
    as

    Download full text from publisher

    File URL: https://e-archivo.uc3m.es/bitstream/handle/10016/6113/we095735.pdf?sequence=5
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Foster, James E & Shorrocks, Anthony F, 1991. "Subgroup Consistent Poverty Indices," Econometrica, Econometric Society, vol. 59(3), pages 687-709, May.
    2. Buhong Zheng, 1997. "Aggregate Poverty Measures," Journal of Economic Surveys, Wiley Blackwell, vol. 11(2), pages 123-162, June.
    3. Katz, J. Sylvan, 1999. "The self-similar science system1," Research Policy, Elsevier, vol. 28(5), pages 501-517, June.
    4. Pedro Albarrán & Juan A. Crespo & Ignacio Ortuño & Javier Ruiz-Castillo, 2011. "The skewness of science in 219 sub-fields and a number of aggregates," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(2), pages 385-397, August.
    5. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "High- and low-impact citation measures: Empirical applications," Journal of Informetrics, Elsevier, vol. 5(1), pages 122-145.
    6. Quesada, Antonio, 2009. "Monotonicity and the Hirsch index," Journal of Informetrics, Elsevier, vol. 3(2), pages 158-160.
    7. Sen, Amartya, 1973. "On Economic Inequality," OUP Catalogue, Oxford University Press, number 9780198281931, Decembrie.
    8. Foster, James & Greer, Joel & Thorbecke, Erik, 1984. "A Class of Decomposable Poverty Measures," Econometrica, Econometric Society, vol. 52(3), pages 761-766, May.
    9. Moed, H. F. & Burger, W. J. M. & Frankfort, J. G. & Van Raan, A. F. J., 1985. "The use of bibliometric data for the measurement of university research performance," Research Policy, Elsevier, vol. 14(3), pages 131-149, June.
    10. Pedro Albarrán & Javier Ruiz‐Castillo, 2011. "References made and citations received by scientific articles," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(1), pages 40-49, January.
    11. Martin, Ben R. & Irvine, John, 1993. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 22(2), pages 106-106, April.
    12. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    13. David Johnson, 1988. "The Measurement of Poverty in Australia: 1981–82 and 1985–86," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 21(3), pages 13-24, September.
    14. Pyatt, Graham, 1987. "Measuring Welfare, Poverty and Inequality," Economic Journal, Royal Economic Society, vol. 97(386), pages 459-467, June.
    15. Jean-Francois Molinari & Alain Molinari, 2008. "A new methodology for ranking scientific institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(1), pages 163-174, April.
    16. Robert J W Tijssen, 2003. "Scoreboards of research excellence," Research Evaluation, Oxford University Press, vol. 12(2), pages 91-103, August.
    17. Sen, Amartya K, 1976. "Poverty: An Ordinal Approach to Measurement," Econometrica, Econometric Society, vol. 44(2), pages 219-231, March.
    18. Satya R. Chakravarty, 2019. "A New Index of Poverty," Themes in Economics, in: Satya R. Chakravarty (ed.), Poverty, Social Exclusion and Stochastic Dominance, pages 31-37, Springer.
    19. Robert J. W. Tijssen & Martijn S. Visser & Thed N. van Leeuwen, 2002. "Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference?," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(3), pages 381-397, July.
    20. Clark, Stephen & Hemming, Richard & Ulph, David, 1981. "On Indices for the Measurement of Poverty," Economic Journal, Royal Economic Society, vol. 91(362), pages 515-526, June.
    21. Thon, Dominique, 1983. "A Note on a Troublesome Axiom for Poverty Indices [On Indices for the Measurement of Poverty]," Economic Journal, Royal Economic Society, vol. 93(369), pages 199-200, March.
    22. Antonio Quesada, 2010. "More axiomatics for the Hirsch index," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 413-418, February.
    23. Zheng, Buhong, 1997. "Aggregate Poverty Measures," Journal of Economic Surveys, Wiley Blackwell, vol. 11(2), pages 123-162, June.
    24. Thon, Dominique, 1979. "On Measuring Poverty," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 25(4), pages 429-439, December.
    25. Albarrán, Pedro & Crespo, Juan A. & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2009. "A comparison of the scientific performance of the U. S. and the European Union at the turn of the XXI century," UC3M Working papers. Economics we095534, Universidad Carlos III de Madrid. Departamento de Economía.
    26. Péter Vinkler, 2003. "Relations of relative scientometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 58(3), pages 687-694, November.
    27. Wolfgang Glänzel & András Schubert & Tibor Braun, 2002. "A relational charting approach to the world of basic research in twelve science fields at the end of the second millennium," Scientometrics, Springer;Akadémiai Kiadó, vol. 55(3), pages 335-348, November.
    28. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    29. Thierry Marchant, 2009. "An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 80(2), pages 325-342, August.
    30. Dominique Thon, 1979. "On Measuring Poverty," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 25(4), pages 429-439, December.
    31. Woeginger, Gerhard J., 2008. "An axiomatic characterization of the Hirsch-index," Mathematical Social Sciences, Elsevier, vol. 56(2), pages 224-232, September.
    32. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    33. J Sylvan Katz, 2000. "Scale-independent indicators and research evaluation," Science and Public Policy, Oxford University Press, vol. 27(1), pages 23-36, February.
    34. Jenkins, Stephen P & Lambert, Peter J, 1997. "Three 'I's of Poverty Curves, with an Analysis of UK Poverty Trends," Oxford Economic Papers, Oxford University Press, vol. 49(3), pages 317-327, July.
    35. Woeginger, Gerhard J., 2008. "A symmetry axiom for scientific impact indices," Journal of Informetrics, Elsevier, vol. 2(4), pages 298-303.
    36. Dasgupta, Partha & Sen, Amartya & Starrett, David, 1973. "Notes on the measurement of inequality," Journal of Economic Theory, Elsevier, vol. 6(2), pages 180-187, April.
    37. Dag W. Aksnes & Gunnar Sivertsen, 2004. "The effect of highly cited papers on national citation indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 59(2), pages 213-224, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Herranz, Neus & Ruiz-Castillo, Javier, 2012. "Sub-field normalization in the multiplicative case: Average-based citation indicators," Journal of Informetrics, Elsevier, vol. 6(4), pages 543-556.
    2. Ricardo Martínez & Antonio Villar, 2018. "Multilevel Proficiency Comparisons With An Application To Educational Outcomes In Pisa," Working Papers 18.04, Universidad Pablo de Olavide, Department of Economics.
    3. Neus Herranz & Javier Ruiz-Castillo, 2013. "The end of the “European Paradox”," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 453-464, April.
    4. Neus Herranz & Javier Ruiz-Castillo, 2012. "Multiplicative and fractional strategies when journals are assigned to several subfields," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(11), pages 2195-2205, November.
    5. Perianes-Rodriguez, Antonio & Ruiz-Castillo, Javier, 2015. "Multiplicative versus fractional counting methods for co-authored publications. The case of the 500 universities in the Leiden Ranking," Journal of Informetrics, Elsevier, vol. 9(4), pages 974-989.
    6. Albarrán, Pedro & Herrero, Carmen & Ruiz-Castillo, Javier & Villar, Antonio, 2017. "The Herrero-Villar approach to citation impact," Journal of Informetrics, Elsevier, vol. 11(2), pages 625-640.
    7. Pedro Albarrán & Ignacio Ortuño & Javier Ruiz-Castillo, 2011. "Average-based versus high- and low-impact indicators for the evaluation of scientific distributions," Research Evaluation, Oxford University Press, vol. 20(4), pages 325-339, October.
    8. Antonio Perianes-Rodriguez & Javier Ruiz-Castillo, 2016. "University citation distributions," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(11), pages 2790-2804, November.
    9. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    10. Neus Herranz & Javier Ruiz-Castillo, 2012. "Sub-field normalization in the multiplicative case: High- and low-impact citation indicators," Research Evaluation, Oxford University Press, vol. 21(2), pages 113-125, April.
    11. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "High- and low-impact citation measures: Empirical applications," Journal of Informetrics, Elsevier, vol. 5(1), pages 122-145.
    12. Antonio Abatemarco & Roberto Dell’Anno, 2013. "Certainty equivalent citation: generalized classes of citation indexes," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(1), pages 263-271, January.
    13. Li, Yunrong & Radicchi, Filippo & Castellano, Claudio & Ruiz-Castillo, Javier, 2013. "Quantitative evaluation of alternative field normalization procedures," Journal of Informetrics, Elsevier, vol. 7(3), pages 746-755.
    14. Javier Ruiz-Castillo, 2012. "The evaluation of citation distributions," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
    15. Perme, Maja Pohar & Stare, Janez & Žaucer, Rok & Žaucer, Matjaž, 2012. "Comparison of the citation distribution and h-index between groups of different sizes," Journal of Informetrics, Elsevier, vol. 6(4), pages 712-720.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. Andrea Bonaccorsi & Tindaro Cicero & Peter Haddawy & Saeed-UL Hassan, 2017. "Explaining the transatlantic gap in research excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 217-241, January.
    18. M. A. Martínez & M. Herrera & E. Contreras & A. Ruíz & E. Herrera-Viedma, 2015. "Characterizing highly cited papers in Social Work through H-Classics," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1713-1729, February.
    19. Antonio Perianes-Rodriguez & Javier Ruiz-Castillo, 2016. "A comparison of two ways of evaluating research units working in different scientific fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 539-561, February.
    20. Bonaccorsi, Andrea & Haddawy, Peter & Cicero, Tindaro & Hassan, Saeed-Ul, 2017. "The solitude of stars. An analysis of the distributed excellence model of European universities," Journal of Informetrics, Elsevier, vol. 11(2), pages 435-454.
    21. M. A. Martínez & M. Herrera & J. López-Gijón & E. Herrera-Viedma, 2014. "H-Classics: characterizing the concept of citation classics through H-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 1971-1983, March.
    22. Albarrán, Pedro & Ruiz-Castillo, Javier, 2012. "The measurement of scientific excellence around the world," UC3M Working papers. Economics we1208, Universidad Carlos III de Madrid. Departamento de Economía.
    23. Argyropoulou, Maria & Soderquist, Klas Eric & Ioannou, George, 2019. "Getting out of the European Paradox trap: Making European research agile and challenge driven," European Management Journal, Elsevier, vol. 37(1), pages 1-5.
    24. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Javier Ruiz-Castillo, 2012. "The evaluation of citation distributions," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 3(1), pages 291-310, March.
    2. Jean–Yves Duclos & Phillipe Grégoire, 2002. "Absolute and Relative Deprivation and the Measurement of Poverty," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 48(4), pages 471-492, December.
    3. Sreenivasan Subramanian, 2004. "Indicators of Inequality and Poverty," WIDER Working Paper Series RP2004-25, World Institute for Development Economic Research (UNU-WIDER).
    4. Albarrán, Pedro & Ortuño, Ignacio & Ruiz-Castillo, Javier, 2011. "High- and low-impact citation measures: Empirical applications," Journal of Informetrics, Elsevier, vol. 5(1), pages 122-145.
    5. James E. Foster & Joel Greer & Erik Thorbecke, 2010. "The Foster-Greer-Thorbecke (FGT) Poverty Measures: Twenty-Five Years Later," Working Papers 2010-14, The George Washington University, Institute for International Economic Policy.
    6. Buhong Zheng, 2007. "Unit-Consistent Poverty Indices," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 31(1), pages 113-142, April.
    7. James Foster & Joel Greer & Erik Thorbecke, 2010. "The Foster–Greer–Thorbecke (FGT) poverty measures: 25 years later," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 8(4), pages 491-524, December.
    8. Zheng, Buhong, 2001. "Statistical inference for poverty measures with relative poverty lines," Journal of Econometrics, Elsevier, vol. 101(2), pages 337-356, April.
    9. Chakravarty, Satya R. & D'Ambrosio, Conchita, 2013. "An axiomatic approach to the measurement of poverty reduction failure," Economic Modelling, Elsevier, vol. 35(C), pages 874-880.
    10. Bibi, Sami & Duclos, Jean-Yves, 2007. "Equity and policy effectiveness with imperfect targeting," Journal of Development Economics, Elsevier, vol. 83(1), pages 109-140, May.
    11. Juan A Crespo & Ignacio Ortuño-Ortín & Javier Ruiz-Castillo, 2012. "The Citation Merit of Scientific Publications," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-9, November.
    12. Zheng, Buhong, 2000. "Minimum Distribution-Sensitivity, Poverty Aversion, and Poverty Orderings," Journal of Economic Theory, Elsevier, vol. 95(1), pages 116-137, November.
    13. Breitmeyer, Carsten & Hakenes, Hendrik & Pfingsten, Andreas, 2004. "From poverty measurement to the measurement of downside risk," Mathematical Social Sciences, Elsevier, vol. 47(3), pages 327-348, May.
    14. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    15. K. Mosler & C. Scheicher, 2008. "Attenuation of relative poverty by taxes and subsidies," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(3), pages 265-283.
    16. Makdissi, Paul & Groleau, Yves, 2002. "Que pouvons-nous apprendre des profils de pauvreté canadiens?," L'Actualité Economique, Société Canadienne de Science Economique, vol. 78(2), pages 257-286, Juin.
    17. Srinivas Goli & Nagendra Kumar Maurya & Moradhvaj & Prem Bhandari, 2019. "Regional Differentials in Multidimensional Poverty in Nepal: Rethinking Dimensions and Method of Computation," SAGE Open, , vol. 9(1), pages 21582440198, March.
    18. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    19. Dipesh Gangopadhyay & Robert B. Nielsen & Velma Zahirovic-Herbert, 2021. "Methodology and Axiomatic Characterization of a Multidimensional and Fuzzy Measure of Deprivation," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 153(1), pages 1-37, January.
    20. Kuan Xu & Zhengxi Lin, 2007. "Participation in Employer-sponsored Training in Canada: Role of Firm Characteristics and Worker Attributes," Working Papers daleconwp2007-02, Dalhousie University, Department of Economics.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cte:werepe:we095735. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Ana Poveda (email available below). General contact details of provider: http://www.eco.uc3m.es/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.