IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v121y2019i3d10.1007_s11192-019-03243-4.html
   My bibliography  Save this article

What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018

Author

Listed:
  • Iman Tahamtan

    (University of Tennessee)

  • Lutz Bornmann

    (Administrative Headquarters of the Max Planck Society)

Abstract

The purpose of this paper is to update the review of Bornmann and Daniel (J Doc 64(1):45–80, 2008) presenting a narrative review of studies on citations in scientific documents. The current review covers 41 studies published between 2006 and 2018. Bornmann and Daniel (2008) focused on earlier years. The current review describes the (new) studies on citation content and context analyses as well as the studies that explore the citation motivation of scholars through surveys or interviews. One focus in this paper is on the technical developments in the last decade, such as the richer meta-data available and machine-readable formats of scientific papers. These developments have resulted in citation context analyses of large datasets in comprehensive studies (which was not possible previously). Many studies in recent years have used computational and machine learning techniques to determine citation functions and polarities, some of which have attempted to overcome the methodological weaknesses of previous studies. The automated recognition of citation functions seems to have the potential to greatly enhance citation indices and information retrieval capabilities. Our review of the empirical studies demonstrates that a paper may be cited for very different scientific and non-scientific reasons. This result accords with the finding by Bornmann and Daniel (2008). The current review also shows that to better understand the relationship between citing and cited documents, a variety of features should be analyzed, primarily the citation context, the semantics and linguistic patterns in citations, citation locations within the citing document, and citation polarity (negative, neutral, positive).

Suggested Citation

  • Iman Tahamtan & Lutz Bornmann, 2019. "What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1635-1684, December.
  • Handle: RePEc:spr:scient:v:121:y:2019:i:3:d:10.1007_s11192-019-03243-4
    DOI: 10.1007/s11192-019-03243-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-019-03243-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-019-03243-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mike Thelwall, 2018. "Does Microsoft Academic find early citations?," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(1), pages 325-334, January.
    2. Shengbo Liu & Chaomei Chen & Kun Ding & Bo Wang & Kan Xu & Yuan Lin, 2014. "Literature retrieval based on citation context," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1293-1307, November.
    3. Saeed-Ul Hassan & Iqra Safder & Anam Akram & Faisal Kamiran, 2018. "A novel machine-learning approach to measuring scientific knowledge flows using citation context analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 973-996, August.
    4. Lutz Bornmann & Robin Haunschild & Sven E. Hug, 2018. "Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 427-437, February.
    5. Marc Bertin & Iana Atanassova & Yves Gingras & Vincent Larivière, 2016. "The invariant distribution of references in scientific articles," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(1), pages 164-177, January.
    6. Metin Doslu & Haluk O. Bingol, 2016. "Context sensitive article ranking with citation context analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 653-671, August.
    7. Björn Hammarfelt, 2011. "Citation analysis on the micro level: The example of Walter Benjamin's Illuminations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(5), pages 819-830, May.
    8. Nigel Harwood, 2008. "Citers' use of citees' names: Findings from a qualitative interview‐based study," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(6), pages 1007-1011, April.
    9. Björn Hammarfelt, 2011. "Citation analysis on the micro level: The example of Walter Benjamin's Illuminations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(5), pages 819-830, May.
    10. Boyack, Kevin W. & van Eck, Nees Jan & Colavizza, Giovanni & Waltman, Ludo, 2018. "Characterizing in-text citations in scientific articles: A large-scale analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 59-73.
    11. Armstrong, J. Scott & Overton, Terry S., 1977. "Estimating Nonresponse Bias in Mail Surveys," MPRA Paper 81694, University Library of Munich, Germany.
    12. Guo Zhang & Ying Ding & Staša Milojević, 2013. "Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(7), pages 1490-1503, July.
    13. Marc Bertin & Iana Atanassova & Cassidy R. Sugimoto & Vincent Lariviere, 2016. "The linguistic patterns and rhetorical structure of citation context: an approach using n-grams," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1417-1434, December.
    14. Chi-Shiou Lin, 2018. "An analysis of citation functions in the humanities and social sciences research from the perspective of problematic citation analysis assumptions," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 797-813, August.
    15. María‐del‐Mar Camacho‐Miñano & Manuel Núñez‐Nickel, 2009. "The multilayered nature of reference selection," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(4), pages 754-777, April.
    16. Tahamtan, Iman & Bornmann, Lutz, 2018. "Creativity in science and the link to cited references: Is the creative potential of papers reflected in their cited references?," Journal of Informetrics, Elsevier, vol. 12(3), pages 906-930.
    17. Xiaodan Zhu & Peter Turney & Daniel Lemire & André Vellino, 2015. "Measuring academic influence: Not all citations are equal," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(2), pages 408-427, February.
    18. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    19. Béatrice Milard, 2014. "The social circles behind scientific references: Relationships between citing and cited authors in chemistry publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(12), pages 2459-2468, December.
    20. Kousha, Kayvan & Thelwall, Mike, 2018. "Can Microsoft Academic help to assess the citation impact of academic books?," Journal of Informetrics, Elsevier, vol. 12(3), pages 972-984.
    21. Thelwall, Mike, 2018. "Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 1-9.
    22. Gali Halevi & Henk F. Moed, 2013. "The thematic and conceptual flow of disciplinary research: A citation context analysis of the journal of informetrics, 2007," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(9), pages 1903-1913, September.
    23. Yu-Wei Chang, 2013. "A comparison of citation contexts between natural sciences and social sciences and humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 535-553, August.
    24. Marcelo Alves Ramos & Joabe Gomes Melo & Ulysses Paulino Albuquerque, 2012. "Citation behavior in popular scientific papers: what is behind obscure citations? The case of ethnobotany," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(3), pages 711-719, September.
    25. Guo Zhang & Ying Ding & Staša Milojević, 2013. "Citation content analysis (CCA): A framework for syntactic and semantic analysis of citation content," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(7), pages 1490-1503, July.
    26. Henry Small, 2011. "Interpreting maps of science using citation context sentiments: a preliminary investigation," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(2), pages 373-388, May.
    27. Nigel Harwood, 2008. "Publication outlets and their effect on academic writers’ citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(2), pages 253-265, November.
    28. Small, Henry, 2018. "Characterizing highly cited method and non-method papers using citation contexts: The role of uncertainty," Journal of Informetrics, Elsevier, vol. 12(2), pages 461-480.
    29. Gali Halevi & Henk F. Moed, 2013. "The thematic and conceptual flow of disciplinary research: A citation context analysis of the journal of informetrics, 2007," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 64(9), pages 1903-1913, September.
    30. Iman Tahamtan & Askar Safipour Afshar & Khadijeh Ahamdzadeh, 2016. "Factors affecting number of citations: a comprehensive review of the literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1195-1225, June.
    31. Kousha, Kayvan & Thelwall, Mike & Abdoli, Mahshid, 2018. "Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 287-298.
    32. Malcolm Wright & J. Scott Armstrong, 2008. "The Ombudsman: Verification of Citations: Fawlty Towers of Knowledge?," Interfaces, INFORMS, vol. 38(2), pages 125-139, April.
    33. Small, Henry & Tseng, Hung & Patek, Mike, 2017. "Discovering discoveries: Identifying biomedical discoveries using citation contexts," Journal of Informetrics, Elsevier, vol. 11(1), pages 46-62.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sehrish Iqbal & Saeed-Ul Hassan & Naif Radi Aljohani & Salem Alelyani & Raheel Nawaz & Lutz Bornmann, 2021. "A decade of in-text citation analysis based on natural language processing and machine learning techniques: an overview of empirical studies," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6551-6599, August.
    2. Wang, Shiyun & Mao, Jin & Lu, Kun & Cao, Yujie & Li, Gang, 2021. "Understanding interdisciplinary knowledge integration through citance analysis: A case study on eHealth," Journal of Informetrics, Elsevier, vol. 15(4).
    3. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    4. Lutz Bornmann & K. Brad Wray & Robin Haunschild, 2020. "Citation concept analysis (CCA): a new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by exemplary case studies including classic books by Thomas S. K," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1051-1074, February.
    5. Zhang, Chengzhi & Liu, Lifan & Wang, Yuzhuo, 2021. "Characterizing references from different disciplines: A perspective of citation content analysis," Journal of Informetrics, Elsevier, vol. 15(2).
    6. Lutz Bornmann & Robin Haunschild & Sven E. Hug, 2018. "Visualizing the context of citations referencing papers published by Eugene Garfield: a new type of keyword co-occurrence analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 427-437, February.
    7. Naif Radi Aljohani & Ayman Fayoumi & Saeed-Ul Hassan, 2021. "An in-text citation classification predictive model for a scholarly search system," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5509-5529, July.
    8. Mingyang Wang & Jiaqi Zhang & Shijia Jiao & Xiangrong Zhang & Na Zhu & Guangsheng Chen, 2020. "Important citation identification by exploiting the syntactic and contextual information of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2109-2129, December.
    9. Lyu, Haihua & Bu, Yi & Zhao, Zhenyue & Zhang, Jiarong & Li, Jiang, 2022. "Citation bias in measuring knowledge flow: Evidence from the web of science at the discipline level," Journal of Informetrics, Elsevier, vol. 16(4).
    10. Chaker Jebari & Enrique Herrera-Viedma & Manuel Jesus Cobo, 2021. "The use of citation context to detect the evolution of research topics: a large-scale analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2971-2989, April.
    11. Hamid R. Jamali & Majid Nabavi & Saeid Asadi, 2018. "How video articles are cited, the case of JoVE: Journal of Visualized Experiments," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1821-1839, December.
    12. Shengzhi Huang & Jiajia Qian & Yong Huang & Wei Lu & Yi Bu & Jinqing Yang & Qikai Cheng, 2022. "Disclosing the relationship between citation structure and future impact of a publication," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(7), pages 1025-1042, July.
    13. Kai Nishikawa, 2023. "How and why are citations between disciplines made? A citation context analysis focusing on natural sciences and social sciences and humanities," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 2975-2997, May.
    14. Marc Bertin & Iana Atanassova & Cassidy R. Sugimoto & Vincent Lariviere, 2016. "The linguistic patterns and rhetorical structure of citation context: an approach using n-grams," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1417-1434, December.
    15. Dongqing Lyu & Xuanmin Ruan & Juan Xie & Ying Cheng, 2021. "The classification of citing motivations: a meta-synthesis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 3243-3264, April.
    16. Hei-Chia Wang & Jen-Wei Cheng & Che-Tsung Yang, 2022. "SentCite: a sentence-level citation recommender based on the salient similarity among multiple segments," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2521-2546, May.
    17. Aurora González-Teruel & Francisca Abad-García, 2018. "The influence of Elfreda Chatman’s theories: a citation context analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1793-1819, December.
    18. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 871-906, January.
    19. Small, Henry, 2018. "Characterizing highly cited method and non-method papers using citation contexts: The role of uncertainty," Journal of Informetrics, Elsevier, vol. 12(2), pages 461-480.
    20. Liu, Xiaojuan & Wang, Chenlin & Chen, Dar-Zen & Huang, Mu-Hsuan, 2022. "Exploring perception of retraction based on mentioned status in post-retraction citations," Journal of Informetrics, Elsevier, vol. 16(3).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:121:y:2019:i:3:d:10.1007_s11192-019-03243-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.