IDEAS home Printed from https://ideas.repec.org/a/gam/jftint/v13y2021i2p31-d487801.html
   My bibliography  Save this article

Language Bias in the Google Scholar Ranking Algorithm

Author

Listed:
  • Cristòfol Rovira

    (Department of Communication, Universitat Pompeu Fabra, 08002 Barcelona, Spain
    UPF Barcelona School of Management, Balmes, 134, 08008 Barcelona, Spain)

  • Lluís Codina

    (Department of Communication, Universitat Pompeu Fabra, 08002 Barcelona, Spain
    UPF Barcelona School of Management, Balmes, 134, 08008 Barcelona, Spain)

  • Carlos Lopezosa

    (Department of Communication, Universitat Pompeu Fabra, 08002 Barcelona, Spain)

Abstract

The visibility of academic articles or conference papers depends on their being easily found in academic search engines, above all in Google Scholar. To enhance this visibility, search engine optimization (SEO) has been applied in recent years to academic search engines in order to optimize documents and, thereby, ensure they are better ranked in search pages (i.e., academic search engine optimization or ASEO). To achieve this degree of optimization, we first need to further our understanding of Google Scholar’s relevance ranking algorithm, so that, based on this knowledge, we can highlight or improve those characteristics that academic documents already present and which are taken into account by the algorithm. This study seeks to advance our knowledge in this line of research by determining whether the language in which a document is published is a positioning factor in the Google Scholar relevance ranking algorithm. Here, we employ a reverse engineering research methodology based on a statistical analysis that uses Spearman’s correlation coefficient. The results obtained point to a bias in multilingual searches conducted in Google Scholar with documents published in languages other than in English being systematically relegated to positions that make them virtually invisible. This finding has important repercussions, both for conducting searches and for optimizing positioning in Google Scholar, being especially critical for articles on subjects that are expressed in the same way in English and other languages, the case, for example, of trademarks, chemical compounds, industrial products, acronyms, drugs, diseases, etc.

Suggested Citation

  • Cristòfol Rovira & Lluís Codina & Carlos Lopezosa, 2021. "Language Bias in the Google Scholar Ranking Algorithm," Future Internet, MDPI, vol. 13(2), pages 1-17, January.
  • Handle: RePEc:gam:jftint:v:13:y:2021:i:2:p:31-:d:487801
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1999-5903/13/2/31/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1999-5903/13/2/31/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Farhadi, Hadi & Salehi, Hadi & Md Yunus, Melor & Arezoo, Aghaei Chadegani & Farhadi, Maryam & Fooladi, Masood & Ale Ebrahim, Nader, 2012. "Does it Matter Which Citation Tool is Used to Compare the H-Index of a Group of Highly Cited Researchers?," MPRA Paper 47414, University Library of Munich, Germany, revised Dec 2012.
    2. Anne-Wil Harzing, 2013. "A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners," Scientometrics, Springer;Akadémiai Kiadó, vol. 94(3), pages 1057-1075, March.
    3. Christos Ziakis & Maro Vlachopoulou & Theodosios Kyrkoudis & Makrina Karagkiozidou, 2019. "Important Factors for Improving Google Search Rank," Future Internet, MDPI, vol. 11(2), pages 1-12, January.
    4. Joost C. F. Winter & Amir A. Zadpoor & Dimitra Dodou, 2014. "The expansion of Google Scholar versus Web of Science: a longitudinal study," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1547-1565, February.
    5. Isidro F. Aguillo, 2012. "Is Google Scholar useful for bibliometrics? A webometric analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 343-351, May.
    6. Anne-Wil Harzing, 2014. "A longitudinal study of Google Scholar coverage between 2012 and 2013," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 565-575, January.
    7. Emilio Delgado López-Cózar & Nicolás Robinson-García & Daniel Torres-Salinas, 2014. "The Google scholar experiment: How to index false papers and manipulate bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 446-454, March.
    8. Cristòfol Rovira & Lluís Codina & Frederic Guerrero-Solé & Carlos Lopezosa, 2019. "Ranking by Relevance and Citation Counts, a Comparative Study: Google Scholar, Microsoft Academic, WoS and Scopus," Future Internet, MDPI, vol. 11(9), pages 1-21, September.
    9. Hamid R. Jamali & Majid Nabavi, 2015. "Open access and sources of full-text articles in Google Scholar in different subject fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1635-1651, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Edisa Lozić & Benjamin Štular, 2023. "Fluent but Not Factual: A Comparative Analysis of ChatGPT and Other AI Chatbots’ Proficiency and Originality in Scientific Writing for Humanities," Future Internet, MDPI, vol. 15(10), pages 1-26, October.
    2. Konstantinos I. Roumeliotis & Nikolaos D. Tselikas & Dimitrios K. Nasiopoulos, 2022. "Airlines’ Sustainability Study Based on Search Engine Optimization Techniques and Technologies," Sustainability, MDPI, vol. 14(18), pages 1-23, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cristòfol Rovira & Lluís Codina & Frederic Guerrero-Solé & Carlos Lopezosa, 2019. "Ranking by Relevance and Citation Counts, a Comparative Study: Google Scholar, Microsoft Academic, WoS and Scopus," Future Internet, MDPI, vol. 11(9), pages 1-21, September.
    2. Moed, Henk F. & Bar-Ilan, Judit & Halevi, Gali, 2016. "A new methodology for comparing Google Scholar and Scopus," Journal of Informetrics, Elsevier, vol. 10(2), pages 533-551.
    3. Martin-Martin, Alberto & Orduna-Malea, Enrique & Harzing, Anne-Wil & Delgado López-Cózar, Emilio, 2017. "Can we use Google Scholar to identify highly-cited documents?," Journal of Informetrics, Elsevier, vol. 11(1), pages 152-163.
    4. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    5. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    6. Enrique Orduna-Malea & Selenay Aytac & Clara Y. Tran, 2019. "Universities through the eyes of bibliographic databases: a retroactive growth comparison of Google Scholar, Scopus and Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 433-450, October.
    7. Vivek Kumar Singh & Satya Swarup Srichandan & Hiran H. Lathabai, 2022. "ResearchGate and Google Scholar: how much do they differ in publications, citations and different metrics and why?," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(3), pages 1515-1542, March.
    8. Hamid R. Jamali & Majid Nabavi, 2015. "Open access and sources of full-text articles in Google Scholar in different subject fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1635-1651, December.
    9. Michael Gusenbauer, 2019. "Google Scholar to overshadow them all? Comparing the sizes of 12 academic search engines and bibliographic databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 177-214, January.
    10. Enrique Orduna-Malea & Juan M. Ayllón & Alberto Martín-Martín & Emilio Delgado López-Cózar, 2015. "Methods for estimating the size of Google Scholar," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 931-949, September.
    11. Anne-Wil Harzing & Satu Alakangas, 2016. "Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(2), pages 787-804, February.
    12. Chris Fields, 2015. "Close to the edge: co-authorship proximity of Nobel laureates in Physiology or Medicine, 1991–2010, to cross-disciplinary brokers," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 267-299, April.
    13. Martín-Martín, Alberto & Orduna-Malea, Enrique & Thelwall, Mike & Delgado López-Cózar, Emilio, 2018. "Google Scholar, Web of Science, and Scopus: A systematic comparison of citations in 252 subject categories," Journal of Informetrics, Elsevier, vol. 12(4), pages 1160-1177.
    14. Samreen Ayaz & Muhammad Tanvir Afzal, 2016. "Identification of conversion factor for completing-h index for the field of mathematics," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1511-1524, December.
    15. Ilenia Ascani & Roberta Ciccola & Maria Serena Chiucchi, 2021. "A Structured Literature Review about the Role of Management Accountants in Sustainability Accounting and Reporting," Sustainability, MDPI, vol. 13(4), pages 1-25, February.
    16. John Mingers & Jesse R. O’Hanley & Musbaudeen Okunola, 2017. "Using Google Scholar institutional level data to evaluate the quality of university research," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(3), pages 1627-1643, December.
    17. Enrique Orduña-Malea & Emilio Delgado López-Cózar, 2014. "Google Scholar Metrics evolution: an analysis according to languages," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(3), pages 2353-2367, March.
    18. Andreas Veglis & Dimitrios Giomelakis, 2019. "Search Engine Optimization," Future Internet, MDPI, vol. 12(1), pages 1-2, December.
    19. Alberto Martín-Martín & Enrique Orduna-Malea & Emilio Delgado López-Cózar, 2018. "A novel method for depicting academic disciplines through Google Scholar Citations: The case of Bibliometrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1251-1273, March.
    20. Antonio Cavacini, 2015. "What is the best database for computer science journal articles?," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(3), pages 2059-2071, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jftint:v:13:y:2021:i:2:p:31-:d:487801. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.