IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v129y2024i1d10.1007_s11192-023-04882-4.html
   My bibliography  Save this article

Can we identify prominent scholars using ChatGPT?

Author

Listed:
  • Frode Eika Sandnes

    (Oslo Metropolitan University)

Abstract

It may be tempting to learn about scholars using ChatGPT. To validate ChatGPT for this task a small experiment was conducted based on the 50 most cited researchers at the author’s university. The results show that ChatGPT (GPT-3.5) only recognized an ad-hoc subset of the scholars with no obvious connection to the respective authors’ citation attributes or internet footprint. Moreover, details about the recognized scholars were often erroneous.

Suggested Citation

  • Frode Eika Sandnes, 2024. "Can we identify prominent scholars using ChatGPT?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(1), pages 713-718, January.
    • Handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04882-4
    DOI: 10.1007/s11192-023-04882-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-023-04882-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-023-04882-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Brady D. Lund & Ting Wang & Nishith Reddy Mannuru & Bing Nie & Somipam Shimray & Ziang Wang, 2023. "ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(5), pages 570-581, May.
    2. Chris Stokel-Walker, 2023. "ChatGPT listed as author on research papers: many scientists disapprove," Nature, Nature, vol. 613(7945), pages 620-621, January.
    3. Enrique Orduña-Malea & Álvaro Cabezas-Clavijo, 2023. "ChatGPT and the potential growing of ghost bibliographic references," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5351-5355, September.
    4. Frode Eika Sandnes, 2021. "Everyone onboard? Participation ratios as a metric for research activity assessments within young universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6105-6113, July.
    5. Evelyn Eika & Frode Eika Sandnes, 2022. "Starstruck by journal prestige and citation counts? On students’ bias and perceptions of trustworthiness according to clues in publication references," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6363-6390, November.
    6. Frode Eika Sandnes, 2020. "A simple back-of-the-envelope test for self-citations using Google Scholar author profiles," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(2), pages 1685-1689, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Frode Eika Sandnes, 2025. "Are there too many papers by the same authors within the same conference proceedings? Norms and extremities within the field of human–computer interaction," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1659-1699, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Frode Eika Sandnes, 2025. "Are there too many papers by the same authors within the same conference proceedings? Norms and extremities within the field of human–computer interaction," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(3), pages 1659-1699, March.
    2. Andrew Delios & Rosalie L. Tung & Arjen Witteloostuijn, 2025. "How to intelligently embrace generative AI: the first guardrails for the use of GenAI in IB research," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 56(4), pages 451-460, June.
    3. Arpan Kumar Kar & P. S. Varsha & Shivakami Rajan, 2023. "Unravelling the Impact of Generative Artificial Intelligence (GAI) in Industrial Applications: A Review of Scientific and Grey Literature," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 24(4), pages 659-689, December.
    4. Leonardo Banh & Gero Strobel, 2023. "Generative artificial intelligence," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-17, December.
    5. Hoffmann, Stefan & Lasarov, Wassili & Dwivedi, Yogesh K., 2024. "AI-empowered scale development: Testing the potential of ChatGPT," Technological Forecasting and Social Change, Elsevier, vol. 205(C).
    6. Alin ZAMFIROIU & Denisa VASILE & Daniel SAVU, 2023. "ChatGPT – A Systematic Review of Published Research Papers," Informatica Economica, Academy of Economic Studies - Bucharest, Romania, vol. 27(1), pages 5-16.
    7. Guido Schryen & Mauricio Marrone & Jiaqi Yang, 2025. "Exploring the scope of generative AI in literature review development," Electronic Markets, Springer;IIM University of St. Gallen, vol. 35(1), pages 1-26, December.
    8. Pawe{l} Niszczota & Paul Conway, 2023. "Judgments of research co-created by generative AI: experimental evidence," Papers 2305.11873, arXiv.org.
    9. Amina Badreddine & Hadjira Larbi Cherif, 2023. "ChatGPT in Academic Research: Demonstrating Limitations through Real Practical Examples," Post-Print hal-04379581, HAL.
    10. Frode Eika Sandnes, 2021. "A bibliometric study of human–computer interaction research activity in the Nordic-Baltic Eight countries," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(6), pages 4733-4767, June.
    11. Gemma Rose Guarin & Gena O. Jugar & Joseph T. Gudelos & Nica Anaiza O. Boco & Mercy P. Cantal & Maui April Jane P. Durano & Erika Mae P. Robel & Trisha May F. Tundag & Beatrice D. Mabitad, 2025. "Perspectives, Practices, and Challenges of ChatGPT Utilization at Eastern Visayas State University Teacher Education Department," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 9(1), pages 4164-4185, January.
    12. Kayvan Kousha, 2024. "How is ChatGPT acknowledged in academic publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 129(12), pages 7959-7969, December.
    13. Li-Yuan Huang & Xun Zhang & Qiang Wang & Zhen-Song Chen & Yang Liu, 2024. "Evaluating Media Knowledge Capabilities of Intelligent Search Dialogue Systems: A Case Study of ChatGPT and New Bing," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 15(4), pages 17284-17307, December.
    14. Wang, Wenyang & Luo, Yuping & Ma, Mingrui & Wang, Jinglin & Sui, Cong, 2025. "A novel forecasting framework leveraging large language model and machine learning for methanol price," Energy, Elsevier, vol. 320(C).
    15. Kuldip Singh & Elizabeth Caroline Agustine & Nur Farhani Binti Samasu & Nur Aida Binti Kipli, 2024. "Exploring Students’ Perceptions of ChatGPT: A Study of Public University in Malaysia," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 8(11), pages 1387-1397, November.
    16. Frode Eika Sandnes, 2021. "Everyone onboard? Participation ratios as a metric for research activity assessments within young universities," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6105-6113, July.
    17. Khizar, Hafiz Muhammad Usman & Ashraf, Aqsa & Yuan, Jingbo & Al-Waqfi, Mohammed, 2025. "Insights into ChatGPT adoption (or resistance) in research practices: The behavioral reasoning perspective," Technological Forecasting and Social Change, Elsevier, vol. 215(C).
    18. Michaela Carni & Tamar Gur & Yossi Maaravi, 2024. "Entrepreneurs’ Social Capital in Overcoming Business Challenges: Case Studies of Seven Greentech, Climate Tech and Agritech Startups," Sustainability, MDPI, vol. 16(19), pages 1-26, September.
    19. Janik Ole Wecks & Johannes Voshaar & Benedikt Jost Plate & Jochen Zimmermann, 2024. "Generative AI Usage and Exam Performance," Papers 2404.19699, arXiv.org, revised Nov 2024.
    20. Cordery, Carolyn J. & Goncharenko, Galina & Polzer, Tobias & McConville, Danielle & Belal, Ataur, 2023. "NGOs’ performance, governance, and accountability in the era of digital transformation," The British Accounting Review, Elsevier, vol. 55(5).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:129:y:2024:i:1:d:10.1007_s11192-023-04882-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.