IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v123y2020i2d10.1007_s11192-020-03417-5.html
   My bibliography  Save this article

How much is too much? The difference between research influence and self-citation excess

Author

Listed:
  • Martin Szomszor

    (Clarivate Analytics)

  • David A. Pendlebury

    (Clarivate Analytics)

  • Jonathan Adams

    (Clarivate Analytics
    King’s College London)

Abstract

Citations can be an indicator of publication significance, utility, attention, visibility or short-term impact but analysts need to confirm whether a high citation count for an individual is a genuine reflection of influence or a consequence of extraordinary, even excessive, self-citation. It has recently been suggested there may be increasing misrepresentation of research performance by individuals who self-cite inordinately to achieve scores and win rewards. In this paper we consider self-referencing and self-citing, describe the typical shape of self-citation patterns for carefully curated publication sets authored by 3517 Highly Cited Researchers and quantify the variance in the distribution of self-citation rates within and between all 21 Essential Science Indicators’ fields. We describe both a generic level of median self-referencing rates, common to most fields, and a graphical, distribution-driven assessment of excessive self-citation that demarcates a threshold not dependent on statistical tests or percentiles (since for some fields all values are within a central ‘normal’ range). We describe this graphical procedure for identifying exceptional self-citation rates but emphasize the necessity for expert interpretation of the citation profiles of specific individuals, particularly in fields with atypical self-citation patterns.

Suggested Citation

  • Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
  • Handle: RePEc:spr:scient:v:123:y:2020:i:2:d:10.1007_s11192-020-03417-5
    DOI: 10.1007/s11192-020-03417-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03417-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-020-03417-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Terrence A. Brooks, 1986. "Evidence of complex citer motivations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 37(1), pages 34-36, January.
    2. P. Pichappan & S. Sarasvady, 2002. "The other side of the coin: The intricacies of author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(2), pages 285-290, June.
    3. Seeber, Marco & Cattaneo, Mattia & Meoli, Michele & Malighetti, Paolo, 2019. "Self-citations as strategic response to the use of metrics for career decisions," Research Policy, Elsevier, vol. 48(2), pages 478-491.
    4. Dangzhi Zhao & Andreas Strotmann & Alicia Cappello, 2018. "In†text function of author self†citations: Implications for research evaluation practice," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 949-952, July.
    5. Ramiro H. Gálvez, 2017. "Assessing author self-citation as a mechanism of relevant knowledge diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1801-1812, June.
    6. Emilio Ferrara & Alfonso E. Romero, 2013. "Scientific impact evaluation and the effect of self-citations: Mitigating the bias by discounting the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(11), pages 2332-2339, November.
    7. Shubhanshu Mishra & Brent D Fegley & Jana Diesner & Vetle I Torvik, 2018. "Self-citation is the hallmark of productive authors, of any gender," PLOS ONE, Public Library of Science, vol. 13(9), pages 1-21, September.
    8. Terrence A. Brooks, 1985. "Private acts and public objects: An investigation of citer motivations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 36(4), pages 223-229, July.
    9. Ernesto Gianoli & Marco A. Molina‐Montenegro, 2009. "Insights into the relationship between the h‐index and self‐citations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(6), pages 1283-1285, June.
    10. Yu, Tian & Yu, Guang & Wang, Ming-Yang, 2014. "Classification method for detecting coercive self-citation in journals," Journal of Informetrics, Elsevier, vol. 8(1), pages 123-135.
    11. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    12. Victoria Bakare & Grant Lewison, 2017. "Country over-citation ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1199-1207, November.
    13. Mathieu Leblond, 2012. "Author self-citations in the field of ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 943-953, June.
    14. Iman Tahamtan & Lutz Bornmann, 2019. "What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1635-1684, December.
    15. Lutz Bornmann, 2017. "Is collaboration among scientists related to the citation impact of papers because their quality increases with collaboration? An analysis based on data from F1000Prime and normalized citation scores," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(4), pages 1036-1047, April.
    16. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    17. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2010. "Self-citations at the meso and individual levels: effects of different calculation methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 517-537, March.
    18. Richard J. Ladle & Peter A. Todd & Ana. C. M. Malhado, 2012. "Assessing insularity in global science," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 745-750, December.
    19. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    20. Leif Engqvist & Joachim G. Frommen, 2010. "New insights into the relationship between the h‐index and self‐citations?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1514-1516, July.
    21. Wen-Yau Cathy Lin & Mu-Hsuan Huang, 2012. "The relationship between co-authorship, currency of references and author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 343-360, February.
    22. Dong-Hui Yang & Xin Li & Xiaoxia Sun & Jie Wan, 2016. "Detecting impact factor manipulation with data mining techniques," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1989-2005, December.
    23. Howard D. White, 2001. "Authors as citers over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(2), pages 87-108.
    24. James H. Fowler & Dag W. Aksnes, 2007. "Does self-citation pay?," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(3), pages 427-437, September.
    25. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    26. Jonathan Adams, 2013. "The fourth age of research," Nature, Nature, vol. 497(7451), pages 557-560, May.
    27. Stephen J. Bensman & Lawrence J. Smolinsky & Alexander I. Pudovkin, 2010. "Mean citation rate per article in mathematics journals: Differences from the scientific model," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1440-1463, July.
    28. Bart Thijs & Wolfgang Glänzel, 2006. "The influence of author self-citations on bibliometric meso-indicators. The case of european universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 71-80, January.
    29. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    30. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    31. Dean Hendrix, 2009. "Institutional self-citation rates: A three year study of universities in the United States," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 321-331, November.
    32. Stephen M. Lawani, 1982. "On the Heterogeneity and Classification of Author Self‐Citations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 33(5), pages 281-284, September.
    33. Ibrahim Shehatta & Abdullah M. Al-Rubaish, 2019. "Impact of country self-citations on bibliometric indicators and ranking of most productive countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 775-791, August.
    34. Li Tang & Philip Shapira & Jan Youtie, 2015. "Is there a clubbing effect underlying Chinese research citation Increases?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1923-1932, September.
    35. E. Garfield & I. H. Sher, 1963. "New factors in the evaluation of scientific literature through citation indexing," American Documentation, Wiley Blackwell, vol. 14(3), pages 195-201, July.
    36. Federico Scarpa & Vincenzo Bianco & Luca A. Tagliafico, 2018. "The impact of the national assessment exercises on self-citation rate and publication venue: an empirical investigation on the engineering academic sector in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 997-1022, November.
    37. Michael H. MacRoberts & Barbara R. MacRoberts, 1989. "Problems of citation analysis: A critical review," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 40(5), pages 342-349, September.
    38. Guang Yu & Liang Wang, 2007. "The self-cited rate of scientific journals and the manipulation of their impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(3), pages 321-330, December.
    39. Leif Engqvist & Joachim G. Frommen, 2010. "New insights into the relationship between the h-index and self-citations?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1514-1516, July.
    40. Lev A. Zhivotovsky & Konstantin V. Krutovsky, 2008. "Self-citation can inflate h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(2), pages 373-375, November.
    41. Frandsen, Tove Faber, 2007. "Journal self-citations—Analysing the JIF mechanism," Journal of Informetrics, Elsevier, vol. 1(1), pages 47-58.
    42. Norman Kaplan, 1965. "The norms of citation behavior: Prolegomena to the footnote," American Documentation, Wiley Blackwell, vol. 16(3), pages 179-184, July.
    43. Juan Miguel Campanario, 2011. "Large increases and decreases in journal impact factors in only one year: The effect of journal self-citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 230-235, February.
    44. R. Rousseau, 1999. "Temporal differences in self-citation rates of scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 44(3), pages 521-531, March.
    45. Stephen J. Bensman & Lawrence J. Smolinsky & Alexander I. Pudovkin, 2010. "Mean citation rate per article in mathematics journals: Differences from the scientific model," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1440-1463, July.
    46. Wolfgang Glänzel & Bart Thijs, 2004. "Does co-authorship inflate the share of self-citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 61(3), pages 395-404, November.
    47. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    48. Simoes, Nadia & Crespo, Nuno, 2020. "Self-Citations and scientific evaluation: Leadership, influence, and performance," Journal of Informetrics, Elsevier, vol. 14(1).
    49. Bornmann, Lutz & Adams, Jonathan & Leydesdorff, Loet, 2018. "The negative effects of citing with a national orientation in terms of recognition: National and international citations in natural-sciences papers from Germany, the Netherlands, and the UK," Journal of Informetrics, Elsevier, vol. 12(3), pages 931-949.
    50. Emilio Ferrara & Alfonso E. Romero, 2013. "Scientific impact evaluation and the effect of self‐citations: Mitigating the bias by discounting the h‐index," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(11), pages 2332-2339, November.
    51. Dag W. Aksnes, 2003. "A macro study of self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 235-246, February.
    52. Zaggl, Michael A., 2017. "Manipulation of explicit reputation in innovation and knowledge exchange communities: The example of referencing in science," Research Policy, Elsevier, vol. 46(5), pages 970-983.
    53. Wolfgang Glänzel & Koenraad Debackere & Bart Thijs & András Schubert, 2006. "A concise review on the role of author self-citations in information science, bibliometrics and science policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(2), pages 263-277, May.
    54. Henry Small, 2004. "On the shoulders of Robert Merton: Towards a normative theory of citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(1), pages 71-79, May.
    55. Juan Miguel Campanario, 2011. "Large increases and decreases in journal impact factors in only one year: The effect of journal self‐citations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 230-235, February.
    56. Ken Hyland, 2003. "Self‐citation and self‐reference: Credibility and promotion in academic publication," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(3), pages 251-259, February.
    57. András Schubert & Wolfgang Glänzel & Bart Thijs, 2006. "The weight of author self-citations. A fractional approach to self-citation counting," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 503-514, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Stefano Vercelli & Leonardo Pellicciari & Andrea Croci & Cesare Maria Cornaggia & Francesca Cecchi & Daniele Piscitelli, 2023. "Self-citation behavior within the health allied professions’ scientific sector in Italy: a bibliometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1205-1217, February.
    2. Lauranne Chaignon & Domingo Docampo & Daniel Egret, 2023. "In search of a scientific elite: highly cited researchers (HCR) in France," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(10), pages 5801-5827, October.
    3. Raf Vanderstraeten & Frédéric Vandermoere, 2021. "Inequalities in the growth of Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8635-8651, October.
    4. Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).
    5. Rossello, Giulia & Martinelli, Arianna, 2023. "Breach of academic values and digital deviant behaviour: The case of Sci-Hub," MERIT Working Papers 2023-009, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    6. Latefa Ali Dardas & Malik Sallam & Amanda Woodward & Nadia Sweis & Narjes Sweis & Faleh A. Sawair, 2023. "Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive," Publications, MDPI, vol. 11(1), pages 1-16, January.
    7. Mansour Haghighat & Javad Hayatdavoudi, 2021. "How hot are hot papers? The issue of prolificacy and self-citation stacking," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 565-578, January.
    8. Yangping Zhou, 2021. "Self-citation and citation of top journal publishers and their interpretation in the journal-discipline context," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6013-6040, July.
    9. Hui Li & Weishu Liu, 2020. "Same same but different: self-citations identified through Scopus and Web of Science Core Collection," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2723-2732, September.
    10. Taşkın, Zehra & Doğan, Güleda & Kulczycki, Emanuel & Zuccala, Alesia Ann, 2021. "Self-Citation Patterns of Journals Indexed in the Journal Citation Reports," Journal of Informetrics, Elsevier, vol. 15(4).
    11. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    12. Siler, Kyle & Larivière, Vincent, 2022. "Who games metrics and rankings? Institutional niches and journal impact factor inflation," Research Policy, Elsevier, vol. 51(10).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Abramo, Giovanni & D'Angelo, Ciriaco Andrea & Grilli, Leonardo, 2021. "The effects of citation-based research evaluation schemes on self-citation behavior," Journal of Informetrics, Elsevier, vol. 15(4).
    2. Mansour Haghighat & Javad Hayatdavoudi, 2021. "How hot are hot papers? The issue of prolificacy and self-citation stacking," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 565-578, January.
    3. Mathieu Leblond, 2012. "Author self-citations in the field of ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 943-953, June.
    4. Ibrahim Shehatta & Abdullah M. Al-Rubaish, 2019. "Impact of country self-citations on bibliometric indicators and ranking of most productive countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 775-791, August.
    5. Rabishankar Giri & Sabuj Kumar Chaudhuri, 2021. "Ranking journals through the lens of active visibility," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2189-2208, March.
    6. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2010. "Self-citations at the meso and individual levels: effects of different calculation methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 517-537, March.
    7. Silvio Peroni & Paolo Ciancarini & Aldo Gangemi & Andrea Giovanni Nuzzolese & Francesco Poggi & Valentina Presutti, 2020. "The practice of self-citations: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 253-282, April.
    8. Hui Li & Weishu Liu, 2020. "Same same but different: self-citations identified through Scopus and Web of Science Core Collection," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2723-2732, September.
    9. Juan Miguel Campanario, 2018. "Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations," Publications, MDPI, vol. 6(4), pages 1-15, November.
    10. Tehmina Amjad & Yusra Rehmat & Ali Daud & Rabeeh Ayaz Abbasi, 2020. "Scientific impact of an author and role of self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 915-932, February.
    11. Taşkın, Zehra & Doğan, Güleda & Kulczycki, Emanuel & Zuccala, Alesia Ann, 2021. "Self-Citation Patterns of Journals Indexed in the Journal Citation Reports," Journal of Informetrics, Elsevier, vol. 15(4).
    12. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    13. Dongqing Lyu & Xuanmin Ruan & Juan Xie & Ying Cheng, 2021. "The classification of citing motivations: a meta-synthesis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3243-3264, April.
    14. Liyue Chen & Jielan Ding & Vincent Larivière, 2022. "Measuring the citation context of national self‐references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 671-686, May.
    15. Paulo Henrique Santos Gonçalves & Thiago Gonçalves-Souza & Ulysses Paulino Albuquerque, 2020. "Chronic anthropogenic disturbances in ecology: a bibliometric approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1103-1117, May.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. Dean Hendrix, 2009. "Institutional self-citation rates: A three year study of universities in the United States," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 321-331, November.
    18. Lina Zhou & Uchechukwuka Amadi & Dongsong Zhang, 2020. "Is Self-Citation Biased? An Investigation via the Lens of Citation Polarity, Density, and Location," Information Systems Frontiers, Springer, vol. 22(1), pages 77-90, February.
    19. James Hartley, 2012. "To cite or not to cite: author self-citations and the impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 313-317, August.
    20. Wen-Yau Cathy Lin & Mu-Hsuan Huang, 2012. "The relationship between co-authorship, currency of references and author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 343-360, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:123:y:2020:i:2:d:10.1007_s11192-020-03417-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.