IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i1d10.1007_s11192-023-04882-4.html
   My bibliography  Save this article

Can we identify prominent scholars using ChatGPT?

Author

Listed:
  • Frode Eika Sandnes

    (Oslo Metropolitan University)

Abstract

It may be tempting to learn about scholars using ChatGPT. To validate ChatGPT for this task a small experiment was conducted based on the 50 most cited researchers at the author’s university. The results show that ChatGPT (GPT-3.5) only recognized an ad-hoc subset of the scholars with no obvious connection to the respective authors’ citation attributes or internet footprint. Moreover, details about the recognized scholars were often erroneous.

Suggested Citation

  • Frode Eika Sandnes, 2024. "Can we identify prominent scholars using ChatGPT?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 713-718, January.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04882-4
    DOI: 10.1007/s11192-023-04882-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04882-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04882-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chris Stokel-Walker, 2023. "ChatGPT listed as author on research papers: many scientists disapprove," Nature, Nature, vol. 613(7945), pages 620-621, January.
    2. Enrique Orduña-Malea & Álvaro Cabezas-Clavijo, 2023. "ChatGPT and the potential growing of ghost bibliographic references," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5351-5355, September.
    3. Frode Eika Sandnes, 2020. "A simple back-of-the-envelope test for self-citations using Google Scholar author profiles," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1685-1689, August.
    4. Brady D. Lund & Ting Wang & Nishith Reddy Mannuru & Bing Nie & Somipam Shimray & Ziang Wang, 2023. "ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 570-581, May.
    5. Frode Eika Sandnes, 2021. "Everyone onboard? Participation ratios as a metric for research activity assessments within young universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6105-6113, July.
    6. Evelyn Eika & Frode Eika Sandnes, 2022. "Starstruck by journal prestige and citation counts? On students’ bias and perceptions of trustworthiness according to clues in publication references," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6363-6390, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido Schryen & Mauricio Marrone & Jiaqi Yang, 2025. "Exploring the scope of generative AI in literature review development," Electronic Markets, Springer;IIM University of St. Gallen, vol. 35(1), pages 1-26, December.
    2. Kayvan Kousha, 2024. "How is ChatGPT acknowledged in academic publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7959-7969, December.
    3. Li-Yuan Huang & Xun Zhang & Qiang Wang & Zhen-Song Chen & Yang Liu, 2024. "Evaluating Media Knowledge Capabilities of Intelligent Search Dialogue Systems: A Case Study of ChatGPT and New Bing," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(4), pages 17284-17307, December.
    4. Janik Ole Wecks & Johannes Voshaar & Benedikt Jost Plate & Jochen Zimmermann, 2024. "Generative AI Usage and Exam Performance," Papers 2404.19699, arXiv.org, revised Nov 2024.
    5. Cordery, Carolyn J. & Goncharenko, Galina & Polzer, Tobias & McConville, Danielle & Belal, Ataur, 2023. "NGOs’ performance, governance, and accountability in the era of digital transformation," The British Accounting Review, Elsevier, vol. 55(5).
    6. Peres, Renana & Schreier, Martin & Schweidel, David & Sorescu, Alina, 2023. "On ChatGPT and beyond: How generative artificial intelligence may affect research, teaching, and practice," International Journal of Research in Marketing, Elsevier, vol. 40(2), pages 269-275.
    7. Timm Teubner & Christoph M. Flath & Christof Weinhardt & Wil Aalst & Oliver Hinz, 2023. "Welcome to the Era of ChatGPT et al," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 65(2), pages 95-101, April.
    8. Andrea Baronchelli, 2023. "Shaping New Norms for AI," Papers 2307.08564, arXiv.org, revised Jun 2024.
    9. Raihan, Asif, 2024. "A review of the potential opportunities and challenges of the digital economy for sustainability," Innovation and Green Development, Elsevier, vol. 3(4).
    10. Jörg Garrel & Jana Mayer, 2023. "Artificial Intelligence in studies—use of ChatGPT and AI-based tools among students in Germany," Palgrave Communications, Palgrave Macmillan, vol. 10(1), pages 1-9, December.
    11. Andres Karjus, 2025. "Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligence," Palgrave Communications, Palgrave Macmillan, vol. 12(1), pages 1-18, December.
    12. Rana, Nripendra P. & Pillai, Rajasshrie & Sivathanu, Brijesh & Malik, Nishtha, 2024. "Assessing the nexus of Generative AI adoption, ethical considerations and organizational performance," Technovation, Elsevier, vol. 135(C).
    13. Giordano, Vito & Spada, Irene & Chiarello, Filippo & Fantoni, Gualtiero, 2024. "The impact of ChatGPT on human skills: A quantitative study on twitter data," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    14. Evelyn Eika & Frode Eika Sandnes, 2022. "Starstruck by journal prestige and citation counts? On students’ bias and perceptions of trustworthiness according to clues in publication references," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6363-6390, November.
    15. Junwei Su & Shan Wu & Jinhui Li, 2024. "MTRGL:Effective Temporal Correlation Discerning through Multi-modal Temporal Relational Graph Learning," Papers 2401.14199, arXiv.org, revised Feb 2024.
    16. Arpan Kumar Kar & P. S. Varsha & Shivakami Rajan, 2023. "Unravelling the Impact of Generative Artificial Intelligence (GAI) in Industrial Applications: A Review of Scientific and Grey Literature," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 24(4), pages 659-689, December.
    17. Leonardo Banh & Gero Strobel, 2023. "Generative artificial intelligence," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-17, December.
    18. Hoffmann, Stefan & Lasarov, Wassili & Dwivedi, Yogesh K., 2024. "AI-empowered scale development: Testing the potential of ChatGPT," Technological Forecasting and Social Change, Elsevier, vol. 205(C).
    19. Alin ZAMFIROIU & Denisa VASILE & Daniel SAVU, 2023. "ChatGPT – A Systematic Review of Published Research Papers," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 27(1), pages 5-16.
    20. Pawe{l} Niszczota & Paul Conway, 2023. "Judgments of research co-created by generative AI: experimental evidence," Papers 2305.11873, arXiv.org.

    More about this item

    Keywords

    Evaluating; Research impact; Research visibility; ChatGPT; Large language models; Scientometrics; Bibliometrics;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04882-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.