IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v89y2011i1d10.1007_s11192-011-0449-z.html
   My bibliography  Save this article

A recursive field-normalized bibliometric performance indicator: an application to the field of library and information science

Author

Listed:
  • Ludo Waltman

    (Leiden University)

  • Erjia Yan

    (Indiana University)

  • Nees Jan Eck

    (Leiden University)

Abstract

Two commonly used ideas in the development of citation-based research performance indicators are the idea of normalizing citation counts based on a field classification scheme and the idea of recursive citation weighing (like in PageRank-inspired indicators). We combine these two ideas in a single indicator, referred to as the recursive mean normalized citation score indicator, and we study the validity of this indicator. Our empirical analysis shows that the proposed indicator is highly sensitive to the field classification scheme that is used. The indicator also has a strong tendency to reinforce biases caused by the classification scheme. Based on these observations, we advise against the use of indicators in which the idea of normalization based on a field classification scheme and the idea of recursive citation weighing are combined.

Suggested Citation

  • Ludo Waltman & Erjia Yan & Nees Jan Eck, 2011. "A recursive field-normalized bibliometric performance indicator: an application to the field of library and information science," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 301-314, October.
    • Handle: RePEc:spr:scient:v:89:y:2011:i:1:d:10.1007_s11192-011-0449-z
    DOI: 10.1007/s11192-011-0449-z
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-011-0449-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-011-0449-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Loet Leydesdorff & Lutz Bornmann, 2011. "How fractional counting of citations affects the impact factor: Normalization in terms of differences in citation potentials among fields of science," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 217-229, February.
    2. Christoph Neuhaus & Hans-Dieter Daniel, 2009. "A new reference standard for citation analysis in chemistry and related fields based on the sections of Chemical Abstracts," Scientometrics, Springer;Akadémiai Kiadó, vol. 78(2), pages 219-229, February.
    3. Zitt, Michel, 2010. "Citing-side normalization of journal impact: A robust variant of the Audience Factor," Journal of Informetrics, Elsevier, vol. 4(3), pages 392-406.
    4. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    5. Nees Jan Eck & Ludo Waltman, 2010. "Software survey: VOSviewer, a computer program for bibliometric mapping," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(2), pages 523-538, August.
    6. Chen, P. & Xie, H. & Maslov, S. & Redner, S., 2007. "Finding scientific gems with Google’s PageRank algorithm," Journal of Informetrics, Elsevier, vol. 1(1), pages 8-15.
    7. Loet Leydesdorff & Jung C. Shin, 2011. "How to evaluate universities in terms of their relative citation impacts: Fractional counting of citations and the normalization of differences among disciplines," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(6), pages 1146-1155, June.
    8. Michel Zitt & Henry Small, 2008. "Modifying the journal impact factor by fractional citation weighting: The audience factor," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1856-1860, September.
    9. Ludo Waltman & Nees Jan van Eck, 2010. "The relation between Eigenfactor, audience factor, and influence weight," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1476-1486, July.
    10. Jonathan Adams & Karen Gurney & Louise Jackson, 2008. "Calibrating the zoom — a test of Zitt’s hypothesis," Scientometrics, Springer;Akadémiai Kiadó, vol. 75(1), pages 81-95, April.
    11. Michel Zitt & Suzy Ramanana-Rahary & Elise Bassecoulard, 2005. "Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation," Scientometrics, Springer;Akadémiai Kiadó, vol. 63(2), pages 373-401, April.
    12. Erjia Yan & Ying Ding, 2010. "Weighted citation: An indicator of an article's prestige," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(8), pages 1635-1643, August.
    13. Karol Życzkowski, 2010. "Citation graph, weighted impact factors and performance indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 301-315, October.
    14. Moed, Henk F., 2010. "Measuring contextual citation impact of scientific journals," Journal of Informetrics, Elsevier, vol. 4(3), pages 265-277.
    15. Lundberg, Jonas, 2007. "Lifting the crown—citation z-score," Journal of Informetrics, Elsevier, vol. 1(2), pages 145-154.
    16. Wolfgang Glänzel & András Schubert & Bart Thijs & Koenraad Debackere, 2011. "A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 415-424, May.
    17. González-Pereira, Borja & Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2010. "A new approach to the metric of journals’ scientific prestige: The SJR indicator," Journal of Informetrics, Elsevier, vol. 4(3), pages 379-391.
    18. Waltman, Ludo & van Eck, Nees Jan & Noyons, Ed C.M., 2010. "A unified approach to mapping and clustering of bibliometric networks," Journal of Informetrics, Elsevier, vol. 4(4), pages 629-635.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "Globalised vs averaged: Bias and ranking performance on the author level," Journal of Informetrics, Elsevier, vol. 13(1), pages 299-313.
    2. Eloy López-Meneses & Esteban Vázquez-Cano & Mariana-Daniela González-Zamar & Emilio Abad-Segura, 2020. "Socioeconomic Effects in Cyberbullying: Global Research Trends in the Educational Context," IJERPH, MDPI, vol. 17(12), pages 1-31, June.
    3. Ludo Waltman & Nees Jan Eck, 2013. "Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 699-716, September.
    4. Yuen-Hsien Tseng & Ming-Yueh Tsay, 2013. "Journal clustering of library and information science for subfield delineation using the bibliometric analysis toolkit: CATAR," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 503-528, May.
    5. P. Dorta-González & M. I. Dorta-González, 2013. "Comparing journals from different fields of science and social science through a JCR subject categories normalized impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(2), pages 645-672, May.
    6. Alejandro Sáez-Martín & Antonio M. López-Hernandez & Carmen Caba-Perez, 2017. "Access to public information: a scientometric study of legal versus voluntary transparency in the public sector," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1697-1720, December.
    7. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    8. Eleni Fragkiadaki & Georgios Evangelidis, 2014. "Review of the indirect citations paradigm: theory and practice of the assessment of papers, authors and journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 261-288, May.
    9. Corrêa Jr., Edilson A. & Silva, Filipi N. & da F. Costa, Luciano & Amancio, Diego R., 2017. "Patterns of authors contribution in scientific manuscripts," Journal of Informetrics, Elsevier, vol. 11(2), pages 498-510.
    10. Janne-Tuomas Seppänen & Hanna Värri & Irene Ylönen, 2022. "Co-citation Percentile Rank and JYUcite: a new network-standardized output-level citation influence metric and its implementation using Dimensions API," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3523-3541, June.
    11. Franceschini, Fiorenzo & Galetto, Maurizio & Maisano, Domenico & Mastrogiacomo, Luca, 2013. "An informetric model for the success-index," Journal of Informetrics, Elsevier, vol. 7(1), pages 109-116.
    12. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    13. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    14. Erjia Yan, 2014. "Topic-based Pagerank: toward a topic-level scientific evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 407-437, August.
    15. A. Abrizah & A. Noorhidawati & A. N. Zainab, 2015. "LIS journals categorization in the Journal Citation Report: a stated preference study," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1083-1099, February.
    16. John Rigby & Barbara Jones, 2020. "Bringing the doctoral thesis by published papers to the Social Sciences and the Humanities: A quantitative easing? A small study of doctoral thesis submission rules and practice in two disciplines in ," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1387-1409, August.
    17. Yongjun Zhu & Erjia Yan, 2015. "Dynamic subfield analysis of disciplines: an examination of the trading impact and knowledge diffusion patterns of computer science," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 335-359, July.
    18. Franceschini, Fiorenzo & Maisano, Domenico, 2014. "Sub-field normalization of the IEEE scientific journals based on their connection with Technical Societies," Journal of Informetrics, Elsevier, vol. 8(3), pages 508-533.
    19. Fiorenzo Franceschini & Domenico Maisano & Luca Mastrogiacomo, 2013. "Evaluating research institutions: the potential of the success-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 85-101, July.
    20. Erjia Yan & Ying Ding & Elin K. Jacob, 2012. "Overlaying communities and topics: an analysis on publication networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 499-513, February.
    21. Maja Jokić, 2020. "Productivity, visibility, authorship, and collaboration in library and information science journals: Central and Eastern European authors," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1189-1219, February.
    22. B Ian Hutchins & Xin Yuan & James M Anderson & George M Santangelo, 2016. "Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level," PLOS Biology, Public Library of Science, vol. 14(9), pages 1-25, September.
    23. Alcaide Muñoz, Laura & Rodríguez Bolívar, Manuel Pedro & Garde Sánchez, Raquel, 2014. "Estudio cienciométrico de la investigación en transparencia informativa, participación ciudadana y prestación de servicios públicos mediante la implementación del e-Gobierno," Revista de Contabilidad - Spanish Accounting Review, Elsevier, vol. 17(2), pages 130-142.
    24. Vaccario, Giacomo & Medo, Matúš & Wider, Nicolas & Mariani, Manuel Sebastian, 2017. "Quantifying and suppressing ranking bias in a large citation network," Journal of Informetrics, Elsevier, vol. 11(3), pages 766-782.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Ludo Waltman & Nees Jan Eck, 2013. "Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(3), pages 699-716, September.
    3. Waltman, Ludo & van Eck, Nees Jan, 2013. "A systematic empirical comparison of different approaches for normalizing citation impact indicators," Journal of Informetrics, Elsevier, vol. 7(4), pages 833-849.
    4. Ahlgren, Per & Waltman, Ludo, 2014. "The correlation between citation-based and expert-based assessments of publication channels: SNIP and SJR vs. Norwegian quality assessments," Journal of Informetrics, Elsevier, vol. 8(4), pages 985-996.
    5. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    6. Mingers, John & Leydesdorff, Loet, 2015. "A review of theory and practice in scientometrics," European Journal of Operational Research, Elsevier, vol. 246(1), pages 1-19.
    7. Bouyssou, Denis & Marchant, Thierry, 2016. "Ranking authors using fractional counting of citations: An axiomatic approach," Journal of Informetrics, Elsevier, vol. 10(1), pages 183-199.
    8. Waltman, Ludo & van Eck, Nees Jan & van Leeuwen, Thed N. & Visser, Martijn S., 2013. "Some modifications to the SNIP journal impact indicator," Journal of Informetrics, Elsevier, vol. 7(2), pages 272-285.
    9. Cristian Colliander & Per Ahlgren, 2019. "Comparison of publication-level approaches to ex-post citation normalization," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 283-300, July.
    10. Zhou, Ping & Leydesdorff, Loet, 2011. "Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing," Journal of Informetrics, Elsevier, vol. 5(3), pages 360-368.
    11. Franceschini, Fiorenzo & Maisano, Domenico, 2014. "Sub-field normalization of the IEEE scientific journals based on their connection with Technical Societies," Journal of Informetrics, Elsevier, vol. 8(3), pages 508-533.
    12. M. Zitt, 2011. "Behind citing-side normalization of citations: some properties of the journal impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 329-344, October.
    13. Tolga Yuret, 2018. "Author-weighted impact factor and reference return ratio: can we attain more equality among fields?," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2097-2111, September.
    14. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    15. Michel Zitt, 2012. "The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 485-503, August.
    16. Henk F. Moed, 2016. "Comprehensive indicator comparisons intelligible to non-experts: the case of two SNIP versions," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 51-65, January.
    17. Fiorenzo Franceschini & Maurizio Galetto & Domenico Maisano & Luca Mastrogiacomo, 2012. "The success-index: an alternative approach to the h-index for evaluating an individual’s research output," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 621-641, September.
    18. Tom Z. J. Fu & Qianqian Song & Dah Ming Chiu, 2014. "The academic social network," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 203-239, October.
    19. Wolfgang Glänzel & Henk F. Moed, 2013. "Opinion paper: thoughts and facts on bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(1), pages 381-394, July.
    20. Bornmann, Lutz & Haunschild, Robin, 2016. "Citation score normalized by cited references (CSNCR): The introduction of a new citation impact indicator," Journal of Informetrics, Elsevier, vol. 10(3), pages 875-887.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:89:y:2011:i:1:d:10.1007_s11192-011-0449-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.