IDEAS home Printed from https://ideas.repec.org/a/spr/reaccs/v29y2024i3d10.1007_s11142-024-09833-9.html?utm_source=chatgpt.com

Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries

Author

Listed:
  • Marc Eulerich

    (University of Duisburg-Essen)

  • Aida Sanatizadeh

    (Northern Illinois University)

  • Hamid Vakilzadeh

    (University of Wisconsin – Whitewater)

  • David A. Wood

    (Brigham Young University)

Abstract

ChatGPT frequently appears in the media, with many predicting significant disruptions, especially in the fields of accounting and auditing. Yet research has demonstrated relatively poor performance of ChatGPT on student assessment questions. We extend this research to examine whether more recent ChatGPT models and capabilities can pass major accounting certification exams including the Certified Public Accountant (CPA), Certified Management Accountant (CMA), Certified Internal Auditor (CIA), and Enrolled Agent (EA) certification exams. We find that the ChatGPT 3.5 model cannot pass any exam (average score across all assessments of 53.1%). However, with additional enhancements, ChatGPT can pass all sections of each tested exam: moving to the ChatGPT 4 model improved scores by an average of 16.5%, providing 10-shot training improved scores an additional 6.6%, and allowing the model to use reasoning and acting (e.g., allow ChatGPT to use a calculator and other resources) improved scores an additional 8.9%. After all these improvements, ChatGPT passed all exams with an average score of 85.1%. This high performance indicates that ChatGPT has sufficient capabilities to disrupt the accounting and auditing industries, which we discuss in detail. This research provides practical insights for accounting professionals, investors, and stakeholders on how to adapt and mitigate the potential harms of this technology in accounting and auditing firms.

Suggested Citation

  • Marc Eulerich & Aida Sanatizadeh & Hamid Vakilzadeh & David A. Wood, 2024. "Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries," Review of Accounting Studies, Springer, vol. 29(3), pages 2318-2349, September.
  • Handle: RePEc:spr:reaccs:v:29:y:2024:i:3:d:10.1007_s11142-024-09833-9
    DOI: 10.1007/s11142-024-09833-9
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11142-024-09833-9
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11142-024-09833-9?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Anastassia Fedyk & James Hodson & Natalya Khimich & Tatiana Fedyk, 2022. "Is artificial intelligence improving the audit process?," Review of Accounting Studies, Springer, vol. 27(3), pages 938-985, September.
    2. Margaret H. Christ & Scott A. Emett & Scott L. Summers & David A. Wood, 2021. "Prepare for takeoff: improving asset measurement and audit quality with drone-enabled inventory audit procedures," Review of Accounting Studies, Springer, vol. 26(4), pages 1323-1343, December.
    3. Kreitmeir, David & Raschky, Paul Anton, 2023. "The Unintended Consequences of Censoring Digital Technology - Evidence from Italy's ChatGPT Ban," SocArXiv v3cgs, Center for Open Science.
    4. Huang, Feiqi & Vasarhelyi, Miklos A., 2019. "Applying robotic process automation (RPA) in auditing: A framework," International Journal of Accounting Information Systems, Elsevier, vol. 35(C).
    5. Shiva Rajgopal, 2021. "Integrating Practice into Accounting Research," Management Science, INFORMS, vol. 67(9), pages 5430-5454, September.
    6. Eddy Cardinaels & Stephan Hollander & Brian J. White, 2019. "Automatic summarization of earnings releases: attributes and effects on investors’ judgments," Review of Accounting Studies, Springer, vol. 24(3), pages 860-890, September.
    7. Geerts, Guido L., 2011. "A design science research methodology and its application to accounting information systems research," International Journal of Accounting Information Systems, Elsevier, vol. 12(2), pages 142-151.
    8. Prithwiraj Choudhury & Evan Starr & Rajshree Agarwal, 2020. "Machine learning and human capital complementarities: Experimental evidence on bias mitigation," Strategic Management Journal, Wiley Blackwell, vol. 41(8), pages 1381-1411, August.
    9. Cardinaels, Eddy & Hollander, Stephan & White, Brian, 2019. "Automatic summarization of earnings releases : Attributes and effects on investors’ judgments," Other publications TiSEM 721f64f4-033e-453b-a3e7-2, Tilburg University, School of Economics and Management.
    10. Tyna Eloundou & Sam Manning & Pamela Mishkin & Daniel Rock, 2023. "GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models," Papers 2303.10130, arXiv.org, revised Aug 2023.
    11. Siliang Tong & Nan Jia & Xueming Luo & Zheng Fang, 2021. "The Janus face of artificial intelligence feedback: Deployment versus disclosure effects on employee performance," Strategic Management Journal, Wiley Blackwell, vol. 42(9), pages 1600-1631, September.
    12. Wood, David A. & Achhpilia, Muskan P. & Adams, Mollie T. & Aghazadeh, Sanaz & Akinyele, Kazeem & Akpan, Mfon & Allee, Kristian D. & Allen, Abigail M. & Almer, Elizabeth D. & Ames, Daniel & Arity, Vikt, 2023. "The ChatGPT artificial intelligence chatbot: How well does it answer accounting assessment questions?," Other publications TiSEM b4a29b3e-18a1-4259-a70c-d, Tilburg University, School of Economics and Management.
    13. Kokina, Julia & Blanchette, Shay, 2019. "Early evidence of digital labor in accounting: Innovation with Robotic Process Automation," International Journal of Accounting Information Systems, Elsevier, vol. 35(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jang, Jaehee & Wu, Xiaoying, 2026. "Non-English textual analysis with large language models: Analysts’ use of MD&A sentiment in earnings forecasting," Journal of Contemporary Accounting and Economics, Elsevier, vol. 22(1).
    2. DeRoy, Tiffany L. & Edmonds, Mark A. & Messina, Marena M., 2026. "Harnessing ChatGPT’s potential: A comprehensive guide for professors," Journal of Accounting Education, Elsevier, vol. 73(C).
    3. Albuquerque, Fábio & dos Santos, Paula Gomes, 2026. "Can ChatGPT be a knowledgeable and accurate ethical accountant? Assessing the overall usefulness of ChatGPT’s answers to ethical dilemmas," Journal of Accounting Education, Elsevier, vol. 73(C).
    4. Osama Abouelela & Ahmed Diab & Safaa Saleh, 2025. "The Relationship Between AI and Audit Planning in Emerging Economies: Does Audit Firm Size Matter?," SAGE Open, , vol. 15(4), pages 21582440251, October.
    5. Christian Fieberg & Lars Hornuf & Maximilian Meiler & David J. Streich, 2025. "Using Large Language Models for Financial Advice," CESifo Working Paper Series 11666, CESifo.
    6. Sun, Hao & Zhu, Xiaoqian & Li, Jianping, 2026. "Revealing corporate accounting fraud: From the perspective of individual investors," Research in International Business and Finance, Elsevier, vol. 86(C).
    7. Bradford Levy, 2026. "Caution Ahead: Numerical Reasoning and Look‐Ahead Bias in AI Models," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 64(3), pages 1139-1188, June.
    8. Elizabeth Blankespoor & Ed Dehaan & Qianqian Li, 2026. "Generative AI in Financial Reporting," Journal of Accounting Research, John Wiley & Sons, Ltd., vol. 64(3), pages 1189-1232, June.
    9. repec:bcp:journl:v:10:y:2026:i:6:p:1694-1709 is not listed on IDEAS
    10. Zhou, Aner & Luo, Yan, 2025. "Exploring the impact of generative AI on student learning in accounting," Journal of Accounting Education, Elsevier, vol. 72(C).
    11. Stratopoulos, Theophanis C. & Wang, Victor Xiaoqi, 2025. "Artificial intelligence and accounting research: a framework and agenda," International Journal of Accounting Information Systems, Elsevier, vol. 56(C).
    12. Nicholas J. Hallman & Zachary T. Kowaleski & Anu Puvvada & Jaime J. Schmidt, 2026. "Sophistication in GenAI Use: Field Evidence from a Large Firm," Papers 2608.27364, arXiv.org.
    13. Ali, Hassnian & Zafar, Muhammad Bilal & Aysan, Ahmet Faruk, 2025. "Generative AI in finance: Replicability, methodological contingencies, and future research directions," Finance Research Letters, Elsevier, vol. 86(PF).
    14. Bertomeu, Jeremy & Lin, Yupeng & Liu, Yibin & Ni, Zhenghui, 2025. "The impact of generative AI on information processing: Evidence from the ban of ChatGPT in Italy," Journal of Accounting and Economics, Elsevier, vol. 80(1).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fotoh, Lazarus Elad & Mugwira, Tatenda, 2025. "Exploring Large Language Models in external audits: Implications and ethical considerations," International Journal of Accounting Information Systems, Elsevier, vol. 56(C).
    2. Christoph Riedl & Eric Bogert, 2024. "Who Benefits from AI? Self-Selection, Skill Gap, and the Hidden Costs of AI Feedback," Papers 2409.18660, arXiv.org, revised Apr 2026.
    3. Charles Ham & Rebecca N. Hann & MaryJane Rabier & Wenfeng Wang, 2025. "Auditor Skill Demands and Audit Quality: Evidence from Job Postings," Management Science, INFORMS, vol. 71(7), pages 5805-5829, July.
    4. Lombardi, Danielle R. & Kim, Meehyun & Sipior, Janice C. & Vasarhelyi, Miklos A., 2025. "The increased role of advanced technology and automation in audit: A delphi study," International Journal of Accounting Information Systems, Elsevier, vol. 56(C).
    5. Baik, Bok & Kim, Alex G. & Kim, David S. & Yoon, Sangwon, 2025. "Vocal delivery quality in earnings conference calls," Journal of Accounting and Economics, Elsevier, vol. 80(1).
    6. Bavaresco, Rodrigo Simon & Nesi, Luan Carlos & Victória Barbosa, Jorge Luis & Antunes, Rodolfo Stoffel & da Rosa Righi, Rodrigo & da Costa, Cristiano André & Vanzin, Mariangela & Dornelles, Daniel & J, 2023. "Machine learning-based automation of accounting services: An exploratory case study," International Journal of Accounting Information Systems, Elsevier, vol. 49(C).
    7. Gu, Xiang & He, Xiqiong & Liu, Hao, 2025. "Risk information disclosure effect on IPO pricing efficiency: Evidence from China," International Review of Economics & Finance, Elsevier, vol. 103(C).
    8. Milan Miric & Nan Jia & Kenneth G. Huang, 2023. "Using supervised machine learning for large‐scale classification in management research: The case for identifying artificial intelligence patents," Strategic Management Journal, Wiley Blackwell, vol. 44(2), pages 491-519, February.
    9. Calvino, Flavio & Fontanelli, Luca, 2026. "AI users are not all alike: The characteristics of French firms buying and developing AI," Research Policy, Elsevier, vol. 55(5).
    10. Standaert, Thomas & Andries, Petra, 2026. "Overcoming difficulties in knowledge transfer: Harnessing the power of AI to drive process innovation," Technovation, Elsevier, vol. 149(C).
    11. Li, Yunjian & Song, Yixiao & Sun, Yanming & Zeng, Mingzhuo, 2024. "When do employees learn from artificial intelligence? The moderating effects of perceived enjoyment and task-related complexity," Technology in Society, Elsevier, vol. 77(C).
    12. Xiqiong He & Tonglin Yang & Hao Liu & Xiaowei Wu, 2025. "Idiosyncratic risk disclosure of IPO firms and institutional investors' withdrawals," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 65(1), pages 399-428, March.
    13. Fontanelli, Luca & Guerini, Mattia & Miniaci, Raffaele & Secchi, Angelo, 2025. "Predictive AI and productivity growth dynamics: Evidence from French firms," Journal of Economic Behavior & Organization, Elsevier, vol. 240(C).
    14. Piyush Gulati & Arianna Marchetti & Victoria Sevcenko & Phanish Puranam, 2025. "How Generative AI Adoption Alters the Demand for Cognitive and Social Skills Within Roles: A Skill-Centric Analysis," Papers 2503.09212, arXiv.org, revised Aug 2026.
    15. Shirish Sundaresan & Isin Guler, 2025. "Algorithmic Recommendation Tools and Experiential Learning in Clinical Care," Organization Science, INFORMS, vol. 36(5), pages 1786-1802, September.
    16. Zengin-Karaibrahimoglu, Yasemin & Homroy, Swarnodeep & Lobo, Gerald J., 2026. "What do we know about sustainability reporting assurance quality?," The British Accounting Review, Elsevier, vol. 58(4).
    17. Emilio Abad-Segura & Mariana-Daniela González-Zamar, 2020. "Research Analysis on Emerging Technologies in Corporate Accounting," Mathematics, MDPI, vol. 8(9), pages 1-29, September.
    18. Vitali, Sonia & Giuliani, Marco, 2024. "Emerging digital technologies and auditing firms: Opportunities and challenges," International Journal of Accounting Information Systems, Elsevier, vol. 53(C).
    19. Wei, Lu & Wei, Mingye & Jing, Haozhe & Jing, Zhongbo, 2025. "Annual report tone and bank risk-taking behavior: Evidence from China," Research in International Business and Finance, Elsevier, vol. 77(PA).
    20. Kokina, Julia & Blanchette, Shay & Davenport, Thomas H. & Pachamanova, Dessislava, 2025. "Challenges and opportunities for artificial intelligence in auditing: Evidence from the field," International Journal of Accounting Information Systems, Elsevier, vol. 56(C).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • M4 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting
    • M40 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting - - - General
    • M42 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting - - - Auditing
    • M49 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting - - - Other

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:reaccs:v:29:y:2024:i:3:d:10.1007_s11142-024-09833-9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.