IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v94y2013i3d10.1007_s11192-012-0777-7.html
   My bibliography  Save this article

A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners

Author

Listed:
  • Anne-Wil Harzing

    () (University of Melbourne)

Abstract

Most governmental research assessment exercises do not use citation data for the Social Sciences and Humanities as Web of Science or Scopus coverage in these disciplines is considered to be insufficient. We therefore assess to what extent Google Scholar can be used as an alternative source of citation data. In order to provide a credible alternative, Google Scholar needs to be stable over time, display comprehensive coverage, and provide non-biased comparisons across disciplines. This article assesses these conditions through a longitudinal study of 20 Nobel Prize winners in Chemistry, Economics, Medicine and Physics. Our results indicate that Google Scholar displays considerable stability over time. However, coverage for disciplines that have traditionally been poorly represented in Google Scholar (Chemistry and Physics) is increasing rapidly. Google Scholar’s coverage is also comprehensive; all of the 800 most cited publications by our Nobelists can be located in Google Scholar, although in four cases there are some problems with the results. Finally, we argue that Google Scholar might provide a less biased comparison across disciplines than the Web of Science. The use of Google Scholar might therefore redress the traditionally disadvantaged position of the Social Sciences in citation analysis.

Suggested Citation

  • Anne-Wil Harzing, 2013. "A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1057-1075, March.
  • Handle: RePEc:spr:scient:v:94:y:2013:i:3:d:10.1007_s11192-012-0777-7
    DOI: 10.1007/s11192-012-0777-7
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-012-0777-7
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bornmann, Lutz & Marx, Werner & Schier, Hermann & Rahm, Erhard & Thor, Andreas & Daniel, Hans-Dieter, 2009. "Convergent validity of bibliometric Google Scholar data in the field of chemistry—Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published els," Journal of Informetrics, Elsevier, vol. 3(1), pages 27-35.
    2. Judit Bar-Ilan, 2010. "Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(3), pages 495-506, March.
    3. John Mingers & Evangelia A. E. C. G. Lipitakis, 2010. "Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(2), pages 613-625, November.
    4. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    5. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    6. Massimo Franceschet, 2010. "A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 83(1), pages 243-258, April.
    7. Judit Bar-Ilan, 2008. "Which h-index? — A comparison of WoS, Scopus and Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 257-271, February.
    8. Bar-Ilan, Judit & Levene, Mark & Lin, Ayelet, 2007. "Some measures for comparing citation databases," Journal of Informetrics, Elsevier, vol. 1(1), pages 26-34.
    9. Kayvan Kousha & Mike Thelwall & Somayeh Rezaie, 2011. "Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(11), pages 2147-2164, November.
    10. Clare V. Thornley & Andrea C. Johnson & Alan F. Smeaton & Hyowon Lee, 2011. "The scholarly impact of TRECVid (2003–2009)," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(4), pages 613-627, April.
    11. Lutz Bornmann & Hans-Dieter Daniel, 2005. "Does the h-index for ranking of scientists really work?," Scientometrics, Springer;Akadémiai Kiadó, vol. 65(3), pages 391-392, December.
    12. Kayvan Kousha & Mike Thelwall, 2008. "Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 273-294, February.
    13. Anne-Wil Harzing, 2005. "Australian Research Output in Economics and Business: High Volume, Low Impact?," Australian Journal of Management, Australian School of Business, vol. 30(2), pages 183-200, December.
    14. Liwen Vaughan & Debora Shaw, 2008. "A new look at evidence of scholarly citation in citation indexes and from web sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 74(2), pages 317-330, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. William M. Cockriel & James B. McDonald, 2018. "The influence of dispersion on journal impact measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 609-622, July.
    2. Clément Bosquet & Pierre-Philippe Combes, 2013. "Are academics who publish more also more cited? Individual determinants of publication and citation records," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 831-857, December.
    3. Anne-Wil Harzing, 2014. "A longitudinal study of Google Scholar coverage between 2012 and 2013," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 565-575, January.
    4. William S. Pearson, 2020. "Research article titles in written feedback on English as a second language writing," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 997-1019, May.
    5. Anne-Wil Harzing & Satu Alakangas, 2017. "Microsoft Academic is one year old: the Phoenix is ready to leave the nest," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1887-1894, September.
    6. Jessica Petersen & Fabian Hattke & Rick Vogel, 2017. "Editorial governance and journal impact: a study of management and business journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1593-1614, September.
    7. Peter James Bentley, 2015. "Cross-country differences in publishing productivity of academics in research universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 865-883, January.
    8. Helene Tenzer & Siri Terjesen & Anne-Wil Harzing, 2017. "Language in International Business: A Review and Agenda for Future Research," Management International Review, Springer, vol. 57(6), pages 815-854, December.
    9. Houcemeddine Turki & Mohamed Ali Hadj Taieb & Mohamed Ben Aouicha & Ajith Abraham, 2020. "Nature or Science: what Google Trends says," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1367-1385, August.
    10. Anne-Wil Harzing, 2015. "Health warning: might contain multiple personalities—the problem of homonyms in Thomson Reuters Essential Science Indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 2259-2270, December.
    11. Chris Fields, 2015. "Close to the edge: co-authorship proximity of Nobel laureates in Physiology or Medicine, 1991–2010, to cross-disciplinary brokers," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 267-299, April.
    12. Houcemeddine Turki & Mohamed Ali Hadj Taieb & Mohamed Ben Aouicha & Ajith Abraham, 0. "Nature or Science: what Google Trends says," Scientometrics, Springer;Akadémiai Kiadó, vol. 0, pages 1-19.
    13. Qurat-ul Ain & Hira Riaz & Muhammad Tanvir Afzal, 2019. "Evaluation of h-index and its citation intensity based variants in the field of mathematics," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(1), pages 187-211, April.
    14. Miguel A. García-Pérez, 2015. "Online supplemental information: a sizeable black hole for citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1655-1659, February.
    15. John Mingers & Jesse R. O’Hanley & Musbaudeen Okunola, 2017. "Using Google Scholar institutional level data to evaluate the quality of university research," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1627-1643, December.
    16. Keshra Sangwal, 2013. "Some citation-related characteristics of scientific journals published in individual countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 719-741, December.
    17. Enrique Orduña-Malea & Emilio Delgado López-Cózar, 2014. "Google Scholar Metrics evolution: an analysis according to languages," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 2353-2367, March.
    18. Joost C. F. Winter & Amir A. Zadpoor & Dimitra Dodou, 2014. "The expansion of Google Scholar versus Web of Science: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1547-1565, February.
    19. Alberto Martín-Martín & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2018. "Coverage of highly-cited documents in Google Scholar, Web of Science, and Scopus: a multidisciplinary comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(3), pages 2175-2188, September.
    20. Anne-Wil Harzing & Wilfred Mijnhardt, 2015. "Proof over promise: towards a more inclusive ranking of Dutch academics in Economics & Business," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 727-749, January.
    21. Zhang, Chengzhi & Zhou, Qingqing, 2020. "Assessing books’ depth and breadth via multi-level mining on tables of contents," Journal of Informetrics, Elsevier, vol. 14(2).
    22. Landoni, Matteo, 2020. "Knowledge creation in state-owned enterprises," Structural Change and Economic Dynamics, Elsevier, vol. 53(C), pages 77-85.
    23. Enrique Orduna-Malea & Selenay Aytac & Clara Y. Tran, 2019. "Universities through the eyes of bibliographic databases: a retroactive growth comparison of Google Scholar, Scopus and Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 433-450, October.
    24. Russell Smyth & Vinod Mishra, 2014. "Academic inbreeding and research productivity and impact in Australian law schools," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 583-618, January.
    25. Andrew W Brown & David B Allison, 2014. "Using Crowdsourcing to Evaluate Published Scientific Literature: Methods and Example," PLOS ONE, Public Library of Science, vol. 9(7), pages 1-9, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:94:y:2013:i:3:d:10.1007_s11192-012-0777-7. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Sonal Shukla) or (Springer Nature Abstracting and Indexing). General contact details of provider: http://www.springer.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.