IDEAS home Printed from https://ideas.repec.org/p/bri/uobdis/25-786.html
   My bibliography  Save this paper

ChatGPT as Economics Tutor: Capabilities and Limitations

Author

Listed:
  • Natalie Brose
  • Christian Spielmann
  • Christian Tode

Abstract

Since the public release of ChatGPT in late 2022, the role of Generative AI chatbots in education has been widely debated. While some see their potential as automated tutors, others worry that inaccuracies and hallucinations could harm student learning. Thisstudy assesses ChatGPT models (GPT-3.5, GPT-4o, and o1preview) across important dimensions of student learning by evaluating their capabilities and limitations to serve as a non-interactive, automated tutor. In particular, we analyse performance in two tasks commonly used in principles of economics courses: explaining economic concepts and answering multiple-choice questions. Our findings indicate that newer models generate very accurate responses, although some inaccuracies persist. A key concern is that ChatGPT presents all responses with full confidence, making errors difficult for students to recognize. Furthermore, explanations are often quite narrow, lacking holistic perspectives, and the quality of examples remains poor. Despite these limitations, we argue that ChatGPT can serve as an effective automated tutor for basic, knowledge-based questions—supportingstudents while posing a relatively low risk of misinformation. Educators can hence recommend Generative AI chatbots for student learning, but should teach students the limitations of the technology.

Suggested Citation

  • Natalie Brose & Christian Spielmann & Christian Tode, 2025. "ChatGPT as Economics Tutor: Capabilities and Limitations," Bristol Economics Discussion Papers 25/786, School of Economics, University of Bristol, UK.
  • Handle: RePEc:bri:uobdis:25/786
    as

    Download full text from publisher

    File URL: http://www.bristol.ac.uk/efm/media/workingpapers/working_papers/pdffiles/dp25786.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Walczak Krzysztof & Cellary Wojciech, 2023. "Challenges for higher education in the era of widespread access to Generative AI," Economics and Business Review, Sciendo, vol. 9(2), pages 71-100, April.
    2. Wayne Geerling & G. Dirk Mateer & Jadrian Wooten & Nikhil Damodaran, 2023. "ChatGPT has Aced the Test of Understanding in College Economics: Now What?," The American Economist, Sage Publications, vol. 68(2), pages 233-245, October.
    3. Wood, David A. & Achhpilia, Muskan P. & Adams, Mollie T. & Aghazadeh, Sanaz & Akinyele, Kazeem & Akpan, Mfon & Allee, Kristian D. & Allen, Abigail M. & Almer, Elizabeth D. & Ames, Daniel & Arity, Vikt, 2023. "The ChatGPT artificial intelligence chatbot: How well does it answer accounting assessment questions?," Other publications TiSEM b4a29b3e-18a1-4259-a70c-d, Tilburg University, School of Economics and Management.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ballantine, Joan & Boyce, Gordon & Stoner, Greg, 2024. "A critical review of AI in accounting education: Threat and opportunity," CRITICAL PERSPECTIVES ON ACCOUNTING, Elsevier, vol. 99(C).
    2. Zhisheng Chen, 2025. "Revolutionizing finance with conversational AI: a focus on ChatGPT implementation and challenges," Palgrave Communications, Palgrave Macmillan, vol. 12(1), pages 1-11, December.
    3. Marc Eulerich & Aida Sanatizadeh & Hamid Vakilzadeh & David A. Wood, 2024. "Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries," Review of Accounting Studies, Springer, vol. 29(3), pages 2318-2349, September.
    4. Dong, Mengming Michael & Stratopoulos, Theophanis C. & Wang, Victor Xiaoqi, 2024. "A scoping review of ChatGPT research in accounting and finance," International Journal of Accounting Information Systems, Elsevier, vol. 55(C).
    5. Haugland Sundkvist, Charlotte & Kulset, Ellen M., 2024. "Teaching accounting in the era of ChatGPT – The student perspective," Journal of Accounting Education, Elsevier, vol. 69(C).
    6. Gu, Hanchi & Schreyer, Marco & Moffitt, Kevin & Vasarhelyi, Miklos, 2024. "Artificial intelligence co-piloted auditing," International Journal of Accounting Information Systems, Elsevier, vol. 54(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bri:uobdis:25/786. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Vicky Jackson (email available below). General contact details of provider: https://edirc.repec.org/data/sebriuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.