IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0138237.html
   My bibliography  Save this article

The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching

Author

Listed:
  • Neal Robert Haddaway
  • Alexandra Mary Collins
  • Deborah Coughlin
  • Stuart Kirk

Abstract

Google Scholar (GS), a commonly used web-based academic search engine, catalogues between 2 and 100 million records of both academic and grey literature (articles not formally published by commercial academic publishers). Google Scholar collates results from across the internet and is free to use. As a result it has received considerable attention as a method for searching for literature, particularly in searches for grey literature, as required by systematic reviews. The reliance on GS as a standalone resource has been greatly debated, however, and its efficacy in grey literature searching has not yet been investigated. Using systematic review case studies from environmental science, we investigated the utility of GS in systematic reviews and in searches for grey literature. Our findings show that GS results contain moderate amounts of grey literature, with the majority found on average at page 80. We also found that, when searched for specifically, the majority of literature identified using Web of Science was also found using GS. However, our findings showed moderate/poor overlap in results when similar search strings were used in Web of Science and GS (10–67%), and that GS missed some important literature in five of six case studies. Furthermore, a general GS search failed to find any grey literature from a case study that involved manual searching of organisations’ websites. If used in systematic reviews for grey literature, we recommend that searches of article titles focus on the first 200 to 300 results. We conclude that whilst Google Scholar can find much grey literature and specific, known studies, it should not be used alone for systematic review searches. Rather, it forms a powerful addition to other traditional search methods. In addition, we advocate the use of tools to transparently document and catalogue GS search results to maintain high levels of transparency and the ability to be updated, critical to systematic reviews.

Suggested Citation

  • Neal Robert Haddaway & Alexandra Mary Collins & Deborah Coughlin & Stuart Kirk, 2015. "The Role of Google Scholar in Evidence Reviews and Its Applicability to Grey Literature Searching," PLOS ONE, Public Library of Science, vol. 10(9), pages 1-17, September.
    • Handle: RePEc:plo:pone00:0138237
    DOI: 10.1371/journal.pone.0138237
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0138237
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0138237&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0138237?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peder Olesen Larsen & Markus Ins, 2010. "The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 575-603, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tuan V. Nguyen & Ly T. Pham, 2011. "Scientific output and its relationship to knowledge economy: an analysis of ASEAN countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 107-117, October.
    2. Ruhua Huang & Yuting Huang & Fan Qi & Leyi Shi & Baiyang Li & Wei Yu, 2022. "Exploring the characteristics of special issues: distribution, topicality, and citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5233-5256, September.
    3. Craig Aaen-Stockdale, 2017. "Selfish Memes: An Update of Richard Dawkins’ Bibliometric Analysis of Key Papers in Sociobiology," Publications, MDPI, vol. 5(2), pages 1-9, May.
    4. Galt, Ryan E. & Pinzón, Natalia & Robinson, Nicholas Ian & Baukloh Coronil, Marcela Beatriz, 2024. "Agroecology and the social sciences: A half-century systematic review," Agricultural Systems, Elsevier, vol. 216(C).
    5. Marie-Violaine Tatry & Dominique Fournier & Benoît Jeannequin & Françoise Dosba, 2014. "EU27 and USA leadership in fruit and vegetable research: a bibliometric study from 2000 to 2009," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 2207-2222, March.
    6. Ju Wen & Lei Lei, 2022. "Adjectives and adverbs in life sciences across 50 years: implications for emotions and readability in academic texts," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(8), pages 4731-4749, August.
    7. Daniele Fanelli, 2012. "Negative results are disappearing from most disciplines and countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 891-904, March.
    8. Mark J. McCabe & Christopher M. Snyder, 2018. "Open Access as a Crude Solution to a Hold‐Up Problem in the Two‐Sided Market for Academic Journals," Journal of Industrial Economics, Wiley Blackwell, vol. 66(2), pages 301-349, June.
    9. Sebastian Vogl & Thomas Scherndl & Anton Kühberger, 2018. "#Psychology: a bibliometric analysis of psychological literature in the online media," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1253-1269, June.
    10. João M. Fernandes, 2014. "Authorship trends in software engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 257-271, October.
    11. Laetitia H. M. Schmitt & Hilary M. Graham & Piran C. L. White, 2016. "Economic Evaluations of the Health Impacts of Weather-Related Extreme Events: A Scoping Review," IJERPH, MDPI, vol. 13(11), pages 1-19, November.
    12. Soo Jeung Lee & Christian Schneijderberg & Yangson Kim & Isabel Steinhardt, 2021. "Have Academics’ Citation Patterns Changed in Response to the Rise of World University Rankings? A Test Using First-Citation Speeds," Sustainability, MDPI, vol. 13(17), pages 1-19, August.
    13. Vanessa Ioannoni & Tommaso Vitale & Corrado Costa & Iris Elliott, 2020. "Depicting communities of Romani studies: on the who, when and where of Roma related scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(3), pages 1473-1490, March.
    14. Beáta Gavurová & Martina Halásková & Samuel Koróny, 2019. "Research and Development Indicators of EU28 Countries from Viewpoint of Super-efficiency DEA Analysis," Acta Universitatis Agriculturae et Silviculturae Mendelianae Brunensis, Mendel University Press, vol. 67(1), pages 225-242.
    15. Fahimnia, Behnam & Sarkis, Joseph & Davarzani, Hoda, 2015. "Green supply chain management: A review and bibliometric analysis," International Journal of Production Economics, Elsevier, vol. 162(C), pages 101-114.
    16. Gil-Clavel, Sofia & Wagenblast, Thorid & Filatova, Tatiana, 2023. "Farmers’ Incremental and Transformational Climate Change Adaptation in Different Regions: A Natural Language Processing Comparative Literature Review," SocArXiv 3dp5e, Center for Open Science.
    17. Xin Gu & Karen Blackmore, 2017. "Quantitative study on Australian academic science," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1009-1035, November.
    18. Anna Tietze & Philip Hofmann, 2019. "The h-index and multi-author hm-index for individual researchers in condensed matter physics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 171-185, April.
    19. Joseph Pozsgai-Alvarez & Iván Pastor Sanz, 2021. "Mapping the (anti-)corruption field: key topics and changing trends, 1968–2020," Journal of Computational Social Science, Springer, vol. 4(2), pages 851-881, November.
    20. Andrada Elena Urda-Cîmpean & Sorana D. Bolboacă & Andrei Achimaş-Cadariu & Tudor Cătălin Drugan, 2016. "Knowledge Production in Two Types of Medical PhD Routes—What’s to Gain?," Publications, MDPI, vol. 4(2), pages 1-16, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0138237. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.