IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0143108.html
   My bibliography  Save this article

The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model

Author

Listed:
  • João A G Moreira
  • Xiao Han T Zeng
  • Luís A Nunes Amaral

Abstract

How to quantify the impact of a researcher’s or an institution’s body of work is a matter of increasing importance to scientists, funding agencies, and hiring committees. The use of bibliometric indicators, such as the h-index or the Journal Impact Factor, have become widespread despite their known limitations. We argue that most existing bibliometric indicators are inconsistent, biased, and, worst of all, susceptible to manipulation. Here, we pursue a principled approach to the development of an indicator to quantify the scientific impact of both individual researchers and research institutions grounded on the functional form of the distribution of the asymptotic number of citations. We validate our approach using the publication records of 1,283 researchers from seven scientific and engineering disciplines and the chemistry departments at the 106 U.S. research institutions classified as “very high research activity”. Our approach has three distinct advantages. First, it accurately captures the overall scientific impact of researchers at all career stages, as measured by asymptotic citation counts. Second, unlike other measures, our indicator is resistant to manipulation and rewards publication quality over quantity. Third, our approach captures the time-evolution of the scientific impact of research institutions.

Suggested Citation

  • João A G Moreira & Xiao Han T Zeng & Luís A Nunes Amaral, 2015. "The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-17, November.
  • Handle: RePEc:plo:pone00:0143108
    DOI: 10.1371/journal.pone.0143108
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0143108
    Download Restriction: no

    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0143108&type=printable
    Download Restriction: no

    File URL: https://libkey.io/10.1371/journal.pone.0143108?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Michael J. Stringer & Marta Sales‐Pardo & Luís A. Nunes Amaral, 2010. "Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1377-1385, July.
    2. Alison Abbott & David Cyranoski & Nicola Jones & Brendan Maher & Quirin Schiermeier & Richard Van Noorden, 2010. "Metrics: Do metrics matter?," Nature, Nature, vol. 465(7300), pages 860-862, June.
    3. Terrence A. Brooks, 1986. "Evidence of complex citer motivations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 37(1), pages 34-36, January.
    4. Michael J. Stringer & Marta Sales-Pardo & Luís A. Nunes Amaral, 2010. "Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1377-1385, July.
    5. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    6. Michael J Stringer & Marta Sales-Pardo & Luís A Nunes Amaral, 2008. "Effectiveness of Journal Ranking Schemes as a Tool for Locating Information," PLOS ONE, Public Library of Science, vol. 3(2), pages 1-8, February.
    7. Quentin L. Burrell, 2003. "Predicting future citation behavior," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(5), pages 372-378, March.
    8. Sune Lehmann & Andrew D. Jackson & Benny E. Lautrup, 2006. "Measures for measures," Nature, Nature, vol. 444(7122), pages 1003-1004, December.
    9. Roberta Kwok, 2013. "Research impact: Altmetrics make their mark," Nature, Nature, vol. 500(7463), pages 491-493, August.
    10. Richard Van Noorden & Brendan Maher & Regina Nuzzo, 2014. "The top 100 papers," Nature, Nature, vol. 514(7524), pages 550-553, October.
    11. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    12. Michael H. MacRoberts & Barbara R. MacRoberts, 1989. "Problems of citation analysis: A critical review," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 40(5), pages 342-349, September.
    13. Haustein, Stefanie & Siebenlist, Tobias, 2011. "Applying social bookmarking data to evaluate journal usage," Journal of Informetrics, Elsevier, vol. 5(3), pages 446-457.
    14. Daniel E. Acuna & Stefano Allesina & Konrad P. Kording, 2012. "Predicting scientific success," Nature, Nature, vol. 489(7415), pages 201-202, September.
    15. Declan Butler, 2013. "Investigating journals: The dark side of publishing," Nature, Nature, vol. 495(7442), pages 433-435, March.
    16. Philip M. Davis & Jason S. Price, 2006. "eJournal interface can influence usage statistics: Implications for libraries, publishers, and Project COUNTER," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(9), pages 1243-1248, July.
    17. Amin Mazloumian, 2012. "Predicting Scholars' Scientific Impact," PLOS ONE, Public Library of Science, vol. 7(11), pages 1-5, November.
    18. Lutz Bornmann & Hans‐Dieter Daniel, 2007. "What do we know about the h index?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(9), pages 1381-1385, July.
    19. Peter Vinkler, 2004. "Characterization of the impact of sets of scientific papers: The Garfield (impact) factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 55(5), pages 431-435, March.
    20. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    21. Quentin L. Burrel, 2001. "Stochastic modelling of the first-citation distribution," Scientometrics, Springer;Akadémiai Kiadó, vol. 52(1), pages 3-12, September.
    22. Jordi Duch & Xiao Han T Zeng & Marta Sales-Pardo & Filippo Radicchi & Shayna Otis & Teresa K Woodruff & Luís A Nunes Amaral, 2012. "The Possible Role of Resource Requirements and Academic Career-Choice Risk on Gender Differences in Publication Rate and Impact," PLOS ONE, Public Library of Science, vol. 7(12), pages 1-11, December.
    23. Amélie Yavchitz & Isabelle Boutron & Aida Bafeta & Ibrahim Marroun & Pierre Charles & Jean Mantz & Philippe Ravaud, 2012. "Misrepresentation of Randomized Controlled Trials in Press Releases and News Coverage: A Cohort Study," PLOS Medicine, Public Library of Science, vol. 9(9), pages 1-11, September.
    24. Fiorenzo Franceschini & Domenico Maisano & Anna Perotti & Andrea Proto, 2010. "Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 203-217, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Filippo Radicchi & Claudio Castellano, 2013. "Analysis of bibliometric indicators for individual scholars in a large data set," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 627-637, December.
    2. Yin, Yian & Wang, Dashun, 2017. "The time dimension of science: Connecting the past to the future," Journal of Informetrics, Elsevier, vol. 11(2), pages 608-621.
    3. David I Stern, 2014. "High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-11, November.
    4. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.
    5. Fiorenzo Franceschini & Domenico Maisano, 2011. "Bibliometric positioning of scientific manufacturing journals: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 463-485, February.
    6. Franceschini, Fiorenzo & Maisano, Domenico, 2010. "The Hirsch spectrum: A novel tool for analyzing scientific journals," Journal of Informetrics, Elsevier, vol. 4(1), pages 64-73.
    7. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    8. Fiorenzo Franceschini & Domenico Maisano & Anna Perotti & Andrea Proto, 2010. "Analysis of the ch-index: an indicator to evaluate the diffusion of scientific research output by citers," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 203-217, October.
    9. Franceschet, Massimo & Costantini, Antonio, 2011. "The first Italian research assessment exercise: A bibliometric perspective," Journal of Informetrics, Elsevier, vol. 5(2), pages 275-291.
    10. Lu Liu & Benjamin F. Jones & Brian Uzzi & Dashun Wang, 2023. "Data, measurement and empirical methods in the science of science," Nature Human Behaviour, Nature, vol. 7(7), pages 1046-1058, July.
    11. John Panaretos & Chrisovaladis Malesios, 2009. "Assessing scientific research performance and impact with single indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(3), pages 635-670, December.
    12. Kaur, Jasleen & Radicchi, Filippo & Menczer, Filippo, 2013. "Universality of scholarly impact metrics," Journal of Informetrics, Elsevier, vol. 7(4), pages 924-932.
    13. Mingkun Wei, 2020. "Research on impact evaluation of open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1027-1049, February.
    14. Bornmann, Lutz & Marx, Werner, 2012. "HistCite analysis of papers constituting the h index research front," Journal of Informetrics, Elsevier, vol. 6(2), pages 285-288.
    15. Anna Tietze & Philip Hofmann, 2019. "The h-index and multi-author hm-index for individual researchers in condensed matter physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 171-185, April.
    16. Chao Lu & Ying Ding & Chengzhi Zhang, 2017. "Understanding the impact change of a highly cited article: a content-based citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 927-945, August.
    17. Li Zhai & Xiangbin Yan & Bin Zhu, 2014. "The H l -index: improvement of H-index based on quality of citing papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1021-1031, February.
    18. Bar-Ilan, Judit, 2008. "Informetrics at the beginning of the 21st century—A review," Journal of Informetrics, Elsevier, vol. 2(1), pages 1-52.
    19. Chen, Meiqian & Guo, Zhaoxia & Dong, Yucheng & Chiclana, Francisco & Herrera-Viedma, Enrique, 2021. "Citations optimal growth path: A tool to analyze sensitivity to citations of h-like indexes," Journal of Informetrics, Elsevier, vol. 15(4).
    20. Lisa Geraci & Steve Balsis & Alexander J. Busch Busch, 2015. "Gender and the h index in psychology," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2023-2034, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0143108. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.