IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v7y2013i3p635-641.html
   My bibliography  Save this article

Counting publications and citations: Is more always better?

Author

Listed:
  • Waltman, Ludo
  • van Eck, Nees Jan
  • Wouters, Paul

Abstract

Is more always better? We address this question in the context of bibliometric indices that aim to assess the scientific impact of individual researchers by counting their number of highly cited publications. We propose a simple model in which the number of citations of a publication depends not only on the scientific impact of the publication but also on other ‘random’ factors. Our model indicates that more need not always be better. It turns out that the most influential researchers may have a systematically lower performance, in terms of highly cited publications, than some of their less influential colleagues. The model also suggests an improved way of counting highly cited publications.

Suggested Citation

  • Waltman, Ludo & van Eck, Nees Jan & Wouters, Paul, 2013. "Counting publications and citations: Is more always better?," Journal of Informetrics, Elsevier, vol. 7(3), pages 635-641.
    • Handle: RePEc:eee:infome:v:7:y:2013:i:3:p:635-641
    DOI: 10.1016/j.joi.2013.04.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157713000357
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2013.04.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Loet Leydesdorff & Lutz Bornmann & Rüdiger Mutz & Tobias Opthof, 2011. "Turning the tables on citation analysis one more time: Principles for comparing sets of documents," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(7), pages 1370-1381, July.
    2. M. V. Simkin & V. P. Roychowdhury, 2005. "Stochastic modeling of citation slips," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(3), pages 367-384, March.
    3. Martin, Ben R. & Irvine, John, 1993. "Assessing basic research : Some partial indicators of scientific progress in radio astronomy," Research Policy, Elsevier, vol. 22(2), pages 106-106, April.
    4. Ludo Waltman & Nees Jan van Eck, 2012. "The inconsistency of the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(2), pages 406-415, February.
    5. Ludo Waltman & Michael Schreiber, 2013. "On the calculation of percentile-based bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 372-379, February.
    6. Martin Ravallion & Adam Wagstaff, 2011. "On measuring scholarly influence by citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(1), pages 321-337, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Les Oxley, 2016. "Elites and Secret Handshakes Versus Metrics and Rule-Based Acclamation: A Comment on "Measuring the Unmeasurable"," Econometric Reviews, Taylor & Francis Journals, vol. 35(1), pages 44-49, January.
    2. Dowling, Grahame R., 2014. "Playing the citations game: From publish or perish to be cited or sidelined," Australasian marketing journal, Elsevier, vol. 22(4), pages 280-287.
    3. Martin Ricker, 2017. "Letter to the Editor: About the quality and impact of scientific articles," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1851-1855, June.
    4. Zahedi, Zohreh & Haustein, Stefanie, 2018. "On the relationships between bibliographic characteristics of scientific documents and citation and Mendeley readership counts: A large-scale analysis of Web of Science publications," Journal of Informetrics, Elsevier, vol. 12(1), pages 191-202.
    5. Danielle H. Lee, 2019. "Predicting the research performance of early career scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1481-1504, December.
    6. van den Besselaar, Peter & Sandström, Ulf, 2015. "Early career grants, performance, and careers: A study on predictive validity of grant decisions," Journal of Informetrics, Elsevier, vol. 9(4), pages 826-838.
    7. Andersen, Jens Peter, 2017. "An empirical and theoretical critique of the Euclidean index," Journal of Informetrics, Elsevier, vol. 11(2), pages 455-465.
    8. Martin Ricker, 2015. "A numerical algorithm with preference statements to evaluate the performance of scientists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 191-212, April.
    9. Morretta, Valentina & Florio, Massimo & Landoni, Matteo, 2023. "The social value of Earth observation: A new evaluation framework for public high-tech infrastructures," Structural Change and Economic Dynamics, Elsevier, vol. 67(C), pages 407-419.
    10. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    11. Letycja Sołoducho-Pelc & Adam Sulich, 2022. "Natural Environment Protection Strategies and Green Management Style: Literature Review," Sustainability, MDPI, vol. 14(17), pages 1-25, August.
    12. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    13. Rousseau, Sandra & Catalano, Giuseppe & Daraio, Cinzia, 2021. "Can we estimate a monetary value of scientific publications?," Research Policy, Elsevier, vol. 50(1).
    14. Morretta, Valentina & Vurchio, Davide & Carrazza, Stefano, 2022. "The socio-economic value of scientific publications: The case of Earth Observation satellites," Technological Forecasting and Social Change, Elsevier, vol. 180(C).
    15. Adam Sulich & Letycja Sołoducho-Pelc, 2022. "Changes in Energy Sector Strategies: A Literature Review," Energies, MDPI, vol. 15(19), pages 1-26, September.
    16. Henk F. Moed & Gali Halevi, 2015. "Multidimensional assessment of scholarly research impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(10), pages 1988-2002, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Schreiber, Michael, 2013. "A case study of the arbitrariness of the h-index and the highly-cited-publications indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 379-387.
    3. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    4. Loet Leydesdorff & Lutz Bornmann & Jonathan Adams, 2019. "The integrated impact indicator revisited (I3*): a non-parametric alternative to the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1669-1694, June.
    5. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    6. Lutz Bornmann & Loet Leydesdorff, 2018. "Count highly-cited papers instead of papers with h citations: use normalized citation counts and compare “like with like”!," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(2), pages 1119-1123, May.
    7. Yves Fassin, 2020. "The HF-rating as a universal complement to the h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 965-990, November.
    8. Albarrán, Pedro & Herrero, Carmen & Ruiz-Castillo, Javier & Villar, Antonio, 2017. "The Herrero-Villar approach to citation impact," Journal of Informetrics, Elsevier, vol. 11(2), pages 625-640.
    9. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    10. Lutz Bornmann & Werner Marx & Andreas Barth, 2013. "The Normalization of Citation Counts Based on Classification Systems," Publications, MDPI, vol. 1(2), pages 1-9, August.
    11. Bouyssou, Denis & Marchant, Thierry, 2014. "An axiomatic approach to bibliometric rankings and indices," Journal of Informetrics, Elsevier, vol. 8(3), pages 449-477.
    12. Lutz Bornmann & Alexander Tekles & Loet Leydesdorff, 2019. "How well does I3 perform for impact measurement compared to other bibliometric indicators? The convergent validity of several (field-normalized) indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 1187-1205, May.
    13. Brito, Ricardo & Navarro, Alonso Rodríguez, 2021. "The inconsistency of h-index: A mathematical analysis," Journal of Informetrics, Elsevier, vol. 15(1).
    14. Schreiber, Michael, 2014. "Is the new citation-rank approach P100′ in bibliometrics really new?," Journal of Informetrics, Elsevier, vol. 8(4), pages 997-1004.
    15. Loet Leydesdorff, 2012. "Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 355-365, August.
    16. Marcello D’Agostino & Valentino Dardanoni & Roberto Ghiselli Ricci, 2017. "How to standardize (if you must)," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 825-843, November.
    17. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    18. Torger Möller & Marion Schmidt & Stefan Hornbostel, 2016. "Assessing the effects of the German Excellence Initiative with bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2217-2239, December.
    19. Pislyakov, Vladimir, 2022. "On some properties of medians, percentiles, baselines, and thresholds in empirical bibliometric analysis," Journal of Informetrics, Elsevier, vol. 16(4).
    20. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:7:y:2013:i:3:p:635-641. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.