IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v98y2014i2d10.1007_s11192-013-1089-2.html
   My bibliography  Save this article

The expansion of Google Scholar versus Web of Science: a longitudinal study

Author

Listed:
  • Joost C. F. Winter

    (Delft University of Technology)

  • Amir A. Zadpoor

    (Delft University of Technology)

  • Dimitra Dodou

    (Delft University of Technology)

Abstract

Web of Science (WoS) and Google Scholar (GS) are prominent citation services with distinct indexing mechanisms. Comprehensive knowledge about the growth patterns of these two citation services is lacking. We analyzed the development of citation counts in WoS and GS for two classic articles and 56 articles from diverse research fields, making a distinction between retroactive growth (i.e., the relative difference between citation counts up to mid-2005 measured in mid-2005 and citation counts up to mid-2005 measured in April 2013) and actual growth (i.e., the relative difference between citation counts up to mid-2005 measured in April 2013 and citation counts up to April 2013 measured in April 2013). One of the classic articles was used for a citation-by-citation analysis. Results showed that GS has substantially grown in a retroactive manner (median of 170 % across articles), especially for articles that initially had low citations counts in GS as compared to WoS. Retroactive growth of WoS was small, with a median of 2 % across articles. Actual growth percentages were moderately higher for GS than for WoS (medians of 54 vs. 41 %). The citation-by-citation analysis showed that the percentage of citations being unique in WoS was lower for more recent citations (6.8 % for citations from 1995 and later vs. 41 % for citations from before 1995), whereas the opposite was noted for GS (57 vs. 33 %). It is concluded that, since its inception, GS has shown substantial expansion, and that the majority of recent works indexed in WoS are now also retrievable via GS. A discussion is provided on quantity versus quality of citations, threats for WoS, weaknesses of GS, and implications for literature research and research evaluation.

Suggested Citation

  • Joost C. F. Winter & Amir A. Zadpoor & Dimitra Dodou, 2014. "The expansion of Google Scholar versus Web of Science: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1547-1565, February.
  • Handle: RePEc:spr:scient:v:98:y:2014:i:2:d:10.1007_s11192-013-1089-2
    DOI: 10.1007/s11192-013-1089-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-013-1089-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11192-013-1089-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Marx, Werner & Schier, Hermann & Rahm, Erhard & Thor, Andreas & Daniel, Hans-Dieter, 2009. "Convergent validity of bibliometric Google Scholar data in the field of chemistry—Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published els," Journal of Informetrics, Elsevier, vol. 3(1), pages 27-35.
    2. Nabil Amara & Réjean Landry, 2012. "Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 553-581, December.
    3. Judit Bar-Ilan, 2010. "Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 495-506, March.
    4. John Mingers & Evangelia A. E. C. G. Lipitakis, 2010. "Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 613-625, November.
    5. Susanne Mikki, 2010. "Comparing Google Scholar and ISI Web of Science for Earth Sciences," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 321-331, February.
    6. Anne-Wil Harzing, 2013. "A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1057-1075, March.
    7. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    8. Linda Butler & Martijn S. Visser, 2006. "Extending citation analysis to non-source items," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(2), pages 327-343, February.
    9. Bar-Ilan, Judit & Levene, Mark & Lin, Ayelet, 2007. "Some measures for comparing citation databases," Journal of Informetrics, Elsevier, vol. 1(1), pages 26-34.
    10. Sen, Amartya K, 1974. "On Some Debates in Capital Theory," Economica, London School of Economics and Political Science, vol. 41(163), pages 328-335, August.
    11. Kayvan Kousha & Mike Thelwall, 2008. "Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 273-294, February.
    12. Peder Olesen Larsen & Markus Ins, 2010. "The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(3), pages 575-603, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Antonio Cavacini, 2015. "What is the best database for computer science journal articles?," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2059-2071, March.
    2. Michael Gusenbauer, 2019. "Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 177-214, January.
    3. Stephen A Gallo & Afton S Carpenter & David Irwin & Caitlin D McPartland & Joseph Travis & Sofie Reynders & Lisa A Thompson & Scott R Glisson, 2014. "The Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies," PLOS ONE, Public Library of Science, vol. 9(9), pages 1-9, September.
    4. Hamid R. Jamali & Majid Nabavi, 2015. "Open access and sources of full-text articles in Google Scholar in different subject fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1635-1651, December.
    5. Maik Hesse & Timm Teubner, 2020. "Reputation portability – quo vadis?," Electronic Markets, Springer;IIM University of St. Gallen, vol. 30(2), pages 331-349, June.
    6. Gunnar Friede, 2019. "Why don't we see more action? A metasynthesis of the investor impediments to integrate environmental, social, and governance factors," Business Strategy and the Environment, Wiley Blackwell, vol. 28(6), pages 1260-1282, September.
    7. Vivek Kumar Singh & Prashasti Singh & Mousumi Karmakar & Jacqueline Leta & Philipp Mayr, 2021. "The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 5113-5142, June.
    8. Jens Jirschitzka & Aileen Oeberst & Richard Göllner & Ulrike Cress, 2017. "Inter-rater reliability and validity of peer reviews in an interdisciplinary field," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1059-1092, November.
    9. Li, Kai & Chen, Pei-Ying & Yan, Erjia, 2019. "Challenges of measuring software impact through citations: An examination of the lme4 R package," Journal of Informetrics, Elsevier, vol. 13(1), pages 449-461.
    10. Enrique Orduna-Malea & Juan M. Ayllón & Alberto Martín-Martín & Emilio Delgado López-Cózar, 2015. "Methods for estimating the size of Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 931-949, September.
    11. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    12. Qurat-ul Ain & Hira Riaz & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its citation intensity based variants in the field of mathematics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 187-211, April.
    13. Muhammad Salman & Mohammad Masroor Ahmed & Muhammad Tanvir Afzal, 2021. "Assessment of author ranking indices based on multi-authorship," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4153-4172, May.
    14. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.
    15. Massimo Aria & Michelangelo Misuraca & Maria Spano, 2020. "Mapping the Evolution of Social Research and Data Science on 30 Years of Social Indicators Research," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 149(3), pages 803-831, June.
    16. Karakaya, Emrah & Nuur, Cali, 2018. "Social sciences and the mining sector: Some insights into recent research trends," Resources Policy, Elsevier, vol. 58(C), pages 257-267.
    17. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    18. Günter Krampen & Peter Weiland & Jürgen Wiesenhütter, 2015. "Citation success of different publication types: a case study on all references in psychology publications from the German-speaking countries (D–A–CH–L–L) in 2009, 2010, and 2011," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 827-840, September.
    19. Muhammad Raheel & Samreen Ayaz & Muhammad Tanvir Afzal, 2018. "Evaluation of h-index, its variants and extensions based on publication age & citation intensity in civil engineering," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1107-1127, March.
    20. Enrique Orduna-Malea & Selenay Aytac & Clara Y. Tran, 2019. "Universities through the eyes of bibliographic databases: a retroactive growth comparison of Google Scholar, Scopus and Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 433-450, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Halevi, Gali & Moed, Henk & Bar-Ilan, Judit, 2017. "Suitability of Google Scholar as a source of scientific information and as a source of data for scientific evaluation—Review of the Literature," Journal of Informetrics, Elsevier, vol. 11(3), pages 823-834.
    2. Anne-Wil Harzing, 2013. "A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1057-1075, March.
    3. Alberto Martín-Martín & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2018. "Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2175-2188, September.
    4. Mehdi Rhaiem & Nabil Amara, 2020. "Determinants of research efficiency in Canadian business schools: evidence from scholar-level data," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 53-99, October.
    5. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    6. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.
    7. Isidro F. Aguillo, 2012. "Is Google Scholar useful for bibliometrics? A webometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 343-351, May.
    8. Antonio Cavacini, 2015. "What is the best database for computer science journal articles?," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2059-2071, March.
    9. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    10. Mike Thelwall, 2017. "Judit Bar-Ilan: information scientist, computer scientist, scientometrician," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1235-1244, December.
    11. Lutz Bornmann & Werner Marx, 2014. "How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 487-509, January.
    12. Clément Bosquet & Pierre-Philippe Combes, 2013. "Are academics who publish more also more cited? Individual determinants of publication and citation records," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 831-857, December.
    13. Pesta, Bryan J., 2018. "Bibliometric analysis across eight years 2008–2015 of Intelligence articles: An updating of Wicherts (2009)," Intelligence, Elsevier, vol. 67(C), pages 26-32.
    14. Martin-Martin, Alberto & Orduna-Malea, Enrique & Harzing, Anne-Wil & Delgado López-Cózar, Emilio, 2017. "Can we use Google Scholar to identify highly-cited documents?," Journal of Informetrics, Elsevier, vol. 11(1), pages 152-163.
    15. Fiorenzo Franceschini & Domenico Maisano, 2011. "Bibliometric positioning of scientific manufacturing journals: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 463-485, February.
    16. Amara, Nabil & Olmos-Peñuela, Julia & Fernández-de-Lucio, Ignacio, 2019. "Overcoming the “lost before translation” problem: An exploratory study," Research Policy, Elsevier, vol. 48(1), pages 22-36.
    17. John Mingers & Jesse R. O’Hanley & Musbaudeen Okunola, 2017. "Using Google Scholar institutional level data to evaluate the quality of university research," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1627-1643, December.
    18. J. W. Fedderke, 2013. "The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(2), pages 177-206, November.
    19. Keshra Sangwal, 2013. "Some citation-related characteristics of scientific journals published in individual countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 719-741, December.
    20. Danielle H. Lee, 2019. "Predictive power of conference-related factors on citation rates of conference papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 281-304, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:98:y:2014:i:2:d:10.1007_s11192-013-1089-2. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.