IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v19y2025i1s1751157724001305.html
   My bibliography  Save this article

Citation counts and inclusion of references in seven free-access scholarly databases: A comparative analysis

Author

Listed:
  • Delgado-Quirós, Lorena
  • Ortega, José Luis

Abstract

The aim of this study is to examine disparities in citation counts amongst scholarly databases and the reasons that contribute to these differences. A random Crossref sample of >115k DOIs was selected and subsequently searched across six databases (Dimensions, Google Scholar, Microsoft Academic, Scilit, Semantic Scholar and The Lens). In July 2021, citation counts and lists of references were extracted from each database for comparative processing and analysis. The findings indicate that publications in Crossref-based databases (Crossref, Dimensions, Scilit and The Lens) have similar citation counts, while documents in search engines (Google Scholar, Microsoft Academic and Semantic Scholar) have a higher number of citations due to a greater coverage of publications, but also to the integration of web copies. Analysis of references has revealed that Scilit only extracts references with Digital Object Identifiers (DOI) and that Semantic Scholar causes significant problems when it adds references from external web versions. Ultimately, the study has shown that all the databases struggle to index references from books and book chapters, which may be attributable to certain academic publishers. The study concludes with a discussion of the potential effects on research evaluation that may arise from this lack of citations.

Suggested Citation

  • Delgado-Quirós, Lorena & Ortega, José Luis, 2025. "Citation counts and inclusion of references in seven free-access scholarly databases: A comparative analysis," Journal of Informetrics, Elsevier, vol. 19(1).
    • Handle: RePEc:eee:infome:v:19:y:2025:i:1:s1751157724001305
    DOI: 10.1016/j.joi.2024.101618
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157724001305
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2024.101618?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Martin-Martin, Alberto & Orduna-Malea, Enrique & Harzing, Anne-Wil & Delgado López-Cózar, Emilio, 2017. "Can we use Google Scholar to identify highly-cited documents?," Journal of Informetrics, Elsevier, vol. 11(1), pages 152-163.
    2. Philippe Mongeon & Adèle Paul-Hus, 2016. "The journal coverage of Web of Science and Scopus: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 213-228, January.
    3. Michael Gusenbauer, 2019. "Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 177-214, January.
    4. Michael H. MacRoberts & Barbara R. MacRoberts, 1989. "Problems of citation analysis: A critical review," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 40(5), pages 342-349, September.
    5. Éric Archambault & David Campbell & Yves Gingras & Vincent Larivière, 2009. "Comparing bibliometric statistics obtained from the Web of Science and Scopus," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(7), pages 1320-1326, July.
    6. Miguel-Angel Vera-Baceta & Michael Thelwall & Kayvan Kousha, 2019. "Web of Science and Scopus language coverage," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(3), pages 1803-1813, December.
    7. Lorena Delgado‐Quirós & Isidro F. Aguillo & Alberto Martín‐Martín & Emilio Delgado López‐Cózar & Enrique Orduña‐Malea & José Luis Ortega, 2024. "Why are these publications missing? Uncovering the reasons behind the exclusion of documents in free‐access scholarly databases," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(1), pages 43-58, January.
    8. Sven E. Hug & Michael Ochsner & Martin P. Brändle, 2017. "Citation analysis with microsoft academic," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 371-378, April.
    9. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 871-906, January.
    10. Kayvan Kousha & Mike Thelwall, 2008. "Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 273-294, February.
    11. Anne-Wil Harzing, 2019. "Two new kids on the block: How do Crossref and Dimensions compare with Google Scholar, Microsoft Academic, Scopus and the Web of Science?," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(1), pages 341-349, July.
    12. Alberto Martín-Martín & Mike Thelwall & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2021. "Correction to: Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: a multidisciplinary comparison of coverage via citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 907-908, January.
    13. Mark P. Carpenter & Francis Narin, 1981. "The adequacy of the science citation index (SCI) as an indicator of international scientific activity," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 32(6), pages 430-439, November.
    14. López-Illescas, Carmen & de Moya-Anegón, Félix & Moed, Henk F., 2008. "Coverage and citation impact of oncological journals in the Web of Science and Scopus," Journal of Informetrics, Elsevier, vol. 2(4), pages 304-316.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael Gusenbauer, 2022. "Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2683-2745, May.
    2. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    3. Gabriel Alves Vieira & Jacqueline Leta, 2024. "biblioverlap: an R package for document matching across bibliographic datasets," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4513-4527, July.
    4. Toluwase Asubiaro & Sodiq Onaolapo & David Mills, 2024. "Regional disparities in Web of Science and Scopus journal coverage," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(3), pages 1469-1491, March.
    5. Joselyne Solórzano & Fernando Morante-Carballo & Néstor Montalván-Burbano & Josué Briones-Bitar & Paúl Carrión-Mero, 2022. "A Systematic Review of the Relationship between Geotechnics and Disasters," Sustainability, MDPI, vol. 14(19), pages 1-31, October.
    6. Adriana Ana Maria Davidescu & Margareta-Stela Florescu & Liviu Cosmin Mosora & Mihaela Hrisanta Mosora & Eduard Mihai Manta, 2022. "A Bibliometric Analysis of Research Publications of the Bucharest University of Economic Studies in Time of Pandemics: Implications for Teachers’ Professional Publishing Activity," IJERPH, MDPI, vol. 19(14), pages 1-36, July.
    7. Lorena Delgado‐Quirós & Isidro F. Aguillo & Alberto Martín‐Martín & Emilio Delgado López‐Cózar & Enrique Orduña‐Malea & José Luis Ortega, 2024. "Why are these publications missing? Uncovering the reasons behind the exclusion of documents in free‐access scholarly databases," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 75(1), pages 43-58, January.
    8. Thelwall, Mike, 2018. "Dimensions: A competitor to Scopus and the Web of Science?," Journal of Informetrics, Elsevier, vol. 12(2), pages 430-435.
    9. Mike Thelwall, 2021. "Alternative medicines worth researching? Citation analyses of acupuncture, chiropractic, homeopathy, and osteopathy 1996–2017," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(10), pages 8731-8747, October.
    10. Dušan Nikolić & Dragan Ivanović & Lidija Ivanović, 2024. "An open-source tool for merging data from multiple citation databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 4573-4595, July.
    11. Teddy Lazebnik & Ariel Rosenfeld, 2025. "How lonely or influential is the Lone Wolf? An analysis of individual scholars’ solo-authorship dynamics," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(5), pages 3053-3069, May.
    12. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    13. Abdelkebir Sahid & Yassine Maleh & Karim Ouazzane, 2025. "Changing landscape of fake news research on social media: a bibliometric analysis," Quality & Quantity: International Journal of Methodology, Springer, vol. 59(2), pages 901-954, April.
    14. Irina Gerasimov & Binita KC & Armin Mehrabian & James Acker & Michael P. McGuire, 2024. "Comparison of datasets citation coverage in Google Scholar, Web of Science, Scopus, Crossref, and DataCite," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(7), pages 3681-3704, July.
    15. Fernando Morante-Carballo & Néstor Montalván-Burbano & Maribel Aguilar-Aguilar & Paúl Carrión-Mero, 2022. "A Bibliometric Analysis of the Scientific Research on Artisanal and Small-Scale Mining," IJERPH, MDPI, vol. 19(13), pages 1-29, July.
    16. Casey Eaton & Amanda Banks & Kristin Weger & Bryan Mesmer & Robert Moreland, 2023. "Understanding perceived influencers on project outcomes and quantifying disciplinary similarities in academic literature," Systems Research and Behavioral Science, Wiley Blackwell, vol. 40(3), pages 460-487, May.
    17. Weishu Liu & Rong Ni & Guangyuan Hu, 2024. "Web of Science Core Collection’s coverage expansion: the forgotten Arts & Humanities Citation Index?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(2), pages 933-955, February.
    18. Vivek Kumar Singh & Prashasti Singh & Mousumi Karmakar & Jacqueline Leta & Philipp Mayr, 2021. "The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5113-5142, June.
    19. Mehdi Toloo & Rouhollah Khodabandelou & Amar Oukil, 2022. "A Comprehensive Bibliometric Analysis of Fractional Programming (1965–2020)," Mathematics, MDPI, vol. 10(11), pages 1-21, May.
    20. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:19:y:2025:i:1:s1751157724001305. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.