IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v123y2020i2d10.1007_s11192-020-03417-5.html
   My bibliography  Save this article

How much is too much? The difference between research influence and self-citation excess

Author

Listed:
  • Martin Szomszor

    (Clarivate Analytics)

  • David A. Pendlebury

    (Clarivate Analytics)

  • Jonathan Adams

    () (Clarivate Analytics
    King’s College London)

Abstract

Citations can be an indicator of publication significance, utility, attention, visibility or short-term impact but analysts need to confirm whether a high citation count for an individual is a genuine reflection of influence or a consequence of extraordinary, even excessive, self-citation. It has recently been suggested there may be increasing misrepresentation of research performance by individuals who self-cite inordinately to achieve scores and win rewards. In this paper we consider self-referencing and self-citing, describe the typical shape of self-citation patterns for carefully curated publication sets authored by 3517 Highly Cited Researchers and quantify the variance in the distribution of self-citation rates within and between all 21 Essential Science Indicators’ fields. We describe both a generic level of median self-referencing rates, common to most fields, and a graphical, distribution-driven assessment of excessive self-citation that demarcates a threshold not dependent on statistical tests or percentiles (since for some fields all values are within a central ‘normal’ range). We describe this graphical procedure for identifying exceptional self-citation rates but emphasize the necessity for expert interpretation of the citation profiles of specific individuals, particularly in fields with atypical self-citation patterns.

Suggested Citation

  • Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
  • Handle: RePEc:spr:scient:v:123:y:2020:i:2:d:10.1007_s11192-020-03417-5
    DOI: 10.1007/s11192-020-03417-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03417-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dean Hendrix, 2009. "Institutional self-citation rates: A three year study of universities in the United States," Scientometrics, Springer;Akadémiai Kiadó, vol. 81(2), pages 321-331, November.
    2. Stephen M. Lawani, 1982. "On the Heterogeneity and Classification of Author Self‐Citations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 33(5), pages 281-284, September.
    3. Stephen J. Bensman & Lawrence J. Smolinsky & Alexander I. Pudovkin, 2010. "Mean citation rate per article in mathematics journals: Differences from the scientific model," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1440-1463, July.
    4. Ibrahim Shehatta & Abdullah M. Al-Rubaish, 2019. "Impact of country self-citations on bibliometric indicators and ranking of most productive countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 775-791, August.
    5. Li Tang & Philip Shapira & Jan Youtie, 2015. "Is there a clubbing effect underlying Chinese research citation Increases?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1923-1932, September.
    6. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    7. Wolfgang Glänzel & Bart Thijs, 2004. "Does co-authorship inflate the share of self-citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 61(3), pages 395-404, November.
    8. E. Garfield & I. H. Sher, 1963. "New factors in the evaluation of scientific literature through citation indexing," American Documentation, Wiley Blackwell, vol. 14(3), pages 195-201, July.
    9. Terrence A. Brooks, 1986. "Evidence of complex citer motivations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 37(1), pages 34-36, January.
    10. P. Pichappan & S. Sarasvady, 2002. "The other side of the coin: The intricacies of author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 54(2), pages 285-290, June.
    11. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    12. Seeber, Marco & Cattaneo, Mattia & Meoli, Michele & Malighetti, Paolo, 2019. "Self-citations as strategic response to the use of metrics for career decisions," Research Policy, Elsevier, vol. 48(2), pages 478-491.
    13. Leif Engqvist & Joachim G. Frommen, 2010. "New insights into the relationship between the h‐index and self‐citations?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1514-1516, July.
    14. Federico Scarpa & Vincenzo Bianco & Luca A. Tagliafico, 2018. "The impact of the national assessment exercises on self-citation rate and publication venue: an empirical investigation on the engineering academic sector in Italy," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(2), pages 997-1022, November.
    15. Dangzhi Zhao & Andreas Strotmann & Alicia Cappello, 2018. "In†text function of author self†citations: Implications for research evaluation practice," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 69(7), pages 949-952, July.
    16. Simoes, Nadia & Crespo, Nuno, 2020. "Self-Citations and scientific evaluation: Leadership, influence, and performance," Journal of Informetrics, Elsevier, vol. 14(1).
    17. Wen-Yau Cathy Lin & Mu-Hsuan Huang, 2012. "The relationship between co-authorship, currency of references and author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 343-360, February.
    18. Ramiro H. Gálvez, 2017. "Assessing author self-citation as a mechanism of relevant knowledge diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1801-1812, June.
    19. Michael H. MacRoberts & Barbara R. MacRoberts, 1989. "Problems of citation analysis: A critical review," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 40(5), pages 342-349, September.
    20. Emilio Ferrara & Alfonso E. Romero, 2013. "Scientific impact evaluation and the effect of self-citations: Mitigating the bias by discounting the h-index," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(11), pages 2332-2339, November.
    21. Guang Yu & Liang Wang, 2007. "The self-cited rate of scientific journals and the manipulation of their impact factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 73(3), pages 321-330, December.
    22. Terrence A. Brooks, 1985. "Private acts and public objects: An investigation of citer motivations," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 36(4), pages 223-229, July.
    23. Ernesto Gianoli & Marco A. Molina‐Montenegro, 2009. "Insights into the relationship between the h‐index and self‐citations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(6), pages 1283-1285, June.
    24. Bornmann, Lutz & Adams, Jonathan & Leydesdorff, Loet, 2018. "The negative effects of citing with a national orientation in terms of recognition: National and international citations in natural-sciences papers from Germany, the Netherlands, and the UK," Journal of Informetrics, Elsevier, vol. 12(3), pages 931-949.
    25. Yu, Tian & Yu, Guang & Wang, Ming-Yang, 2014. "Classification method for detecting coercive self-citation in journals," Journal of Informetrics, Elsevier, vol. 8(1), pages 123-135.
    26. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    27. Victoria Bakare & Grant Lewison, 2017. "Country over-citation ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1199-1207, November.
    28. Emilio Ferrara & Alfonso E. Romero, 2013. "Scientific impact evaluation and the effect of self‐citations: Mitigating the bias by discounting the h‐index," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(11), pages 2332-2339, November.
    29. Leif Engqvist & Joachim G. Frommen, 2010. "New insights into the relationship between the h-index and self-citations?," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(7), pages 1514-1516, July.
    30. Dong-Hui Yang & Xin Li & Xiaoxia Sun & Jie Wan, 2016. "Detecting impact factor manipulation with data mining techniques," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1989-2005, December.
    31. Howard D. White, 2001. "Authors as citers over time," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 52(2), pages 87-108.
    32. Mathieu Leblond, 2012. "Author self-citations in the field of ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 943-953, June.
    33. James H. Fowler & Dag W. Aksnes, 2007. "Does self-citation pay?," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(3), pages 427-437, September.
    34. Lev A. Zhivotovsky & Konstantin V. Krutovsky, 2008. "Self-citation can inflate h-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(2), pages 373-375, November.
    35. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    36. Iman Tahamtan & Lutz Bornmann, 2019. "What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1635-1684, December.
    37. Lutz Bornmann, 2017. "Is collaboration among scientists related to the citation impact of papers because their quality increases with collaboration? An analysis based on data from F1000Prime and normalized citation scores," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 68(4), pages 1036-1047, April.
    38. Frandsen, Tove Faber, 2007. "Journal self-citations—Analysing the JIF mechanism," Journal of Informetrics, Elsevier, vol. 1(1), pages 47-58.
    39. Norman Kaplan, 1965. "The norms of citation behavior: Prolegomena to the footnote," American Documentation, Wiley Blackwell, vol. 16(3), pages 179-184, July.
    40. Dag W. Aksnes, 2003. "A macro study of self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 56(2), pages 235-246, February.
    41. Zaggl, Michael A., 2017. "Manipulation of explicit reputation in innovation and knowledge exchange communities: The example of referencing in science," Research Policy, Elsevier, vol. 46(5), pages 970-983.
    42. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2010. "Self-citations at the meso and individual levels: effects of different calculation methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 517-537, March.
    43. Wolfgang Glänzel & Koenraad Debackere & Bart Thijs & András Schubert, 2006. "A concise review on the role of author self-citations in information science, bibliometrics and science policy," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(2), pages 263-277, May.
    44. Stephen J. Bensman & Lawrence J. Smolinsky & Alexander I. Pudovkin, 2010. "Mean citation rate per article in mathematics journals: Differences from the scientific model," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(7), pages 1440-1463, July.
    45. Bart Thijs & Wolfgang Glänzel, 2006. "The influence of author self-citations on bibliometric meso-indicators. The case of european universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 71-80, January.
    46. Richard J. Ladle & Peter A. Todd & Ana. C. M. Malhado, 2012. "Assessing insularity in global science," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 745-750, December.
    47. Henry Small, 2004. "On the shoulders of Robert Merton: Towards a normative theory of citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(1), pages 71-79, May.
    48. Juan Miguel Campanario, 2011. "Large increases and decreases in journal impact factors in only one year: The effect of journal self‐citations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 230-235, February.
    49. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    50. Juan Miguel Campanario, 2011. "Large increases and decreases in journal impact factors in only one year: The effect of journal self-citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 230-235, February.
    51. R. Rousseau, 1999. "Temporal differences in self-citation rates of scientific journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 44(3), pages 521-531, March.
    52. Ken Hyland, 2003. "Self‐citation and self‐reference: Credibility and promotion in academic publication," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(3), pages 251-259, February.
    53. András Schubert & Wolfgang Glänzel & Bart Thijs, 2006. "The weight of author self-citations. A fractional approach to self-citation counting," Scientometrics, Springer;Akadémiai Kiadó, vol. 67(3), pages 503-514, June.
    54. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mansour Haghighat & Javad Hayatdavoudi, 2021. "How hot are hot papers? The issue of prolificacy and self-citation stacking," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 565-578, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mansour Haghighat & Javad Hayatdavoudi, 2021. "How hot are hot papers? The issue of prolificacy and self-citation stacking," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 565-578, January.
    2. Ibrahim Shehatta & Abdullah M. Al-Rubaish, 2019. "Impact of country self-citations on bibliometric indicators and ranking of most productive countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 775-791, August.
    3. Mathieu Leblond, 2012. "Author self-citations in the field of ecology," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 943-953, June.
    4. Rodrigo Costas & Thed N. Leeuwen & María Bordons, 2010. "Self-citations at the meso and individual levels: effects of different calculation methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 517-537, March.
    5. Silvio Peroni & Paolo Ciancarini & Aldo Gangemi & Andrea Giovanni Nuzzolese & Francesco Poggi & Valentina Presutti, 2020. "The practice of self-citations: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(1), pages 253-282, April.
    6. Hui Li & Weishu Liu, 2020. "Same same but different: self-citations identified through Scopus and Web of Science Core Collection," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2723-2732, September.
    7. Hui Li & Weishu Liu, 0. "Same same but different: self-citations identified through Scopus and Web of Science Core Collection," Scientometrics, Springer;Akadémiai Kiadó, vol. 0, pages 1-10.
    8. Lina Zhou & Uchechukwuka Amadi & Dongsong Zhang, 2020. "Is Self-Citation Biased? An Investigation via the Lens of Citation Polarity, Density, and Location," Information Systems Frontiers, Springer, vol. 22(1), pages 77-90, February.
    9. Copiello, Sergio, 2019. "Peer and neighborhood effects: Citation analysis using a spatial autoregressive model and pseudo-spatial data," Journal of Informetrics, Elsevier, vol. 13(1), pages 238-254.
    10. James Hartley, 2012. "To cite or not to cite: author self-citations and the impact factor," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 313-317, August.
    11. Victoria Bakare & Grant Lewison, 2017. "Country over-citation ratios," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1199-1207, November.
    12. Simoes, Nadia & Crespo, Nuno, 2020. "Self-Citations and scientific evaluation: Leadership, influence, and performance," Journal of Informetrics, Elsevier, vol. 14(1).
    13. Petersen, Alexander M. & Pan, Raj K. & Pammolli, Fabio & Fortunato, Santo, 2019. "Methods to account for citation inflation in research evaluation," Research Policy, Elsevier, vol. 48(7), pages 1855-1865.
    14. Michelle L. Dion & Sara McLaughlin Mitchell & Jane L. Sumner, 2020. "Gender, seniority, and self-citation practices in political science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 1-28, October.
    15. Petersen, Alexander M., 2019. "Megajournal mismanagement: Manuscript decision bias and anomalous editor activity at PLOS ONE," Journal of Informetrics, Elsevier, vol. 13(4).
    16. Wen-Yau Cathy Lin & Mu-Hsuan Huang, 2012. "The relationship between co-authorship, currency of references and author self-citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(2), pages 343-360, February.
    17. Stephen Carley & Alan L. Porter & Jan Youtie, 2013. "Toward a more precise definition of self-citation," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(2), pages 777-780, February.
    18. Daniel Teodorescu & Tudorel Andrei, 2014. "An examination of “citation circles” for social sciences journals in Eastern European countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 209-231, May.
    19. Anthony F. J. Raan, 2012. "Properties of journal impact in relation to bibliometric research group performance indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 457-469, August.
    20. Juan Miguel Campanario, 2018. "Journals that Rise from the Fourth Quartile to the First Quartile in Six Years or Less: Mechanisms of Change and the Role of Journal Self-Citations," Publications, MDPI, Open Access Journal, vol. 6(4), pages 1-15, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:123:y:2020:i:2:d:10.1007_s11192-020-03417-5. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sonal Shukla) or (Springer Nature Abstracting and Indexing). General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.