IDEAS home Printed from https://ideas.repec.org/a/eee/ireced/v49y2025ics1477388025000131.html

Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses☆

Author

Listed:
  • Fiedler, Alexandra
  • Döpke, Jörg

Abstract

We investigate whether human experts can identify AI-generated academic texts more accurately than current machine-based detectors. Conducted as a survey experiment at a German university of applied sciences, 63 lecturers in engineering, economics, and social sciences were asked to evaluate short excerpts (200–300 words) from both human-generated and AI-generated texts. These texts varied by discipline and writing level (student vs. professional) with the AI-generated content. The results show that both human evaluators and AI detectors correctly identified AI-generated texts only slightly better than chance, with humans achieving a recognition rate of 57 % for AI texts and 64 % for human-generated texts. There was no statistically significant difference between human and machine performance. Notably, professional-level AI texts were the most difficult to identify, with less than 20 % of respondents correctly classifying them. Regression analyses suggest that prior teaching experience slightly improves recognition accuracy, while subjective judgments of text quality were not influenced by actual or presumed authorship. These findings suggest that current written examination practices are increasingly vulnerable to undetected AI use. Both human judgment and existing AI detectors show high error rates, especially for high-quality AI-generated content. We conclude that a reconsideration of traditional assessment formats in academia is warranted.

Suggested Citation

  • Fiedler, Alexandra & Döpke, Jörg, 2025. "Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses☆," International Review of Economics Education, Elsevier, vol. 49(C).
    • Handle: RePEc:eee:ireced:v:49:y:2025:i:c:s1477388025000131
    DOI: 10.1016/j.iree.2025.100321
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1477388025000131
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.iree.2025.100321?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Anton Korinek, 2023. "Generative AI for Economic Research: Use Cases and Implications for Economists," Journal of Economic Literature, American Economic Association, vol. 61(4), pages 1281-1317, December.
    2. Jaime A. Manalo IV & Girlie Nora A. Abrigo & Rowena P. De Guzman & Perry G. I. Del Rosario & Rhemilyn Z. Relado-Sevilla, 2025. "Understanding family farming in the Philippine context," Development in Practice, Taylor & Francis Journals, vol. 35(4), pages 566-584, May.
    3. Holly Else, 2023. "Abstracts written by ChatGPT fool scientists," Nature, Nature, vol. 613(7944), pages 423-423, January.
    4. repec:iza:izadps:dp17511 is not listed on IDEAS
    5. Alexander Bick & Adam Blandin & David Deming, 2023. "The Rapid Adoption of Generative AI," On the Economy 98843, Federal Reserve Bank of St. Louis.
    6. Xiaqing Hu & Haitao Ma, 2025. "Understanding the resilience of intercity innovation networks," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-14, December.
    7. Sengupta, Atanu & De, Sanjoy, 2025. "Nobel Growth - An Understanding of 2025 Economics Nobel," MPRA Paper 126695, University Library of Munich, Germany.
    8. Stankov, Petar, 2024. "The death of exams? Grade inflation and student satisfaction when coursework replaces exams," International Review of Economics Education, Elsevier, vol. 46(C).
    9. Yvonne Tan & Imaan Bayoumi & Bruce Knox & Michele Cole & Logan Jackson & Autumn Watson & Colleen M Davison & Susan A Bartels & Eva Purkey, 2025. "Understanding social inclusion: A directed content analysis," PLOS ONE, Public Library of Science, vol. 20(10), pages 1-16, October.
    10. Moravec, Vaclav & Hynek, Nik & Skare, Marinko & Gavurova, Beata & Kubak, Matus, 2024. "Human or machine? The perception of artificial intelligence in journalism, its socio-economic conditions, and technological developments toward the digital future," Technological Forecasting and Social Change, Elsevier, vol. 200(C).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bouramdane, Ayat-Allah, 2025. "Optimal strategies to combat land degradation amid climate change: Aligned with UNCCD and Kunming-Montreal commitments," Land Use Policy, Elsevier, vol. 157(C).
    2. Andersen, Jens Peter & Degn, Lise & Fishberg, Rachel & Graversen, Ebbe K. & Horbach, Serge P.J.M. & Schmidt, Evanthia Kalpazidou & Schneider, Jesper W. & Sørensen, Mads P., 2025. "Generative Artificial Intelligence (GenAI) in the research process – A survey of researchers’ practices and perceptions," Technology in Society, Elsevier, vol. 81(C).
    3. Paola Cillo & Gaia Rubera, 2025. "Generative AI in innovation and marketing processes: A roadmap of research opportunities," Journal of the Academy of Marketing Science, Springer, vol. 53(3), pages 684-701, May.
    4. Liu, Yan & Wang, He, 2026. "Who on earth is using Generative AI?," World Development, Elsevier, vol. 199(C).
    5. Giordano, Vito & Spada, Irene & Chiarello, Filippo & Fantoni, Gualtiero, 2024. "The impact of ChatGPT on human skills: A quantitative study on twitter data," Technological Forecasting and Social Change, Elsevier, vol. 203(C).
    6. Jian Xue & Qian Zhang & Wu Zhu, 2025. "Generative AI for Analysts," Papers 2512.19705, arXiv.org.
    7. repec:ces:ceswps:_12072 is not listed on IDEAS
    8. Zara Contractor & Germ'an Reyes, 2025. "Generative AI in Higher Education: Evidence from an Elite College," Papers 2508.00717, arXiv.org, revised Apr 2026.
    9. Edisa Lozić & Benjamin Štular, 2023. "Fluent but Not Factual: A Comparative Analysis of ChatGPT and Other AI Chatbots’ Proficiency and Originality in Scientific Writing for Humanities," Future Internet, MDPI, vol. 15(10), pages 1-26, October.
    10. Iñaki Aldasoro & Ajit Desai, 2025. "Money Talks: AI Agents for Cash Management in Payment Systems," Staff Working Papers 25-35, Bank of Canada.
    11. Francesco Venturini, 2025. "Generative AI and Income Growth: Early Evidence on Global Data," Gospodarka Narodowa. The Polish Journal of Economics, Warsaw School of Economics, issue 3, pages 31-46.
    12. Cristina Angelico & Enrico Bernardini, 2026. "Can GenAI fill banks' emissions data gaps?," Questioni di Economia e Finanza (Occasional Papers) 1003, Bank of Italy, Economic Research and International Relations Area.
    13. Arpan Kumar Kar & P. S. Varsha & Shivakami Rajan, 2023. "Unravelling the Impact of Generative Artificial Intelligence (GAI) in Industrial Applications: A Review of Scientific and Grey Literature," Global Journal of Flexible Systems Management, Springer;Global Institute of Flexible Systems Management, vol. 24(4), pages 659-689, December.
    14. Dragan Filimonovic & Christian Rutzer & Conny Wunsch, 2025. "Can GenAI Improve Academic Performance? Evidence from the Social and Behavioral Sciences," Papers 2510.02408, arXiv.org.
    15. Anastasios Evgenidis & Apostolos Fasianos, 2025. "AI news shocks and the macroeconomy: evidence from UK patent data," IFS Working Papers W25/48, Institute for Fiscal Studies.
    16. Andrew Johnston & Christos A. Makridis, 2026. "AI, Output, and Employment," CESifo Working Paper Series 12579, CESifo.
    17. Kiran Tomlinson & Sonia Jaffe & Will Wang & Scott Counts & Siddharth Suri, 2025. "Working with AI: Measuring the Applicability of Generative AI to Occupations," Papers 2507.07935, arXiv.org, revised Dec 2025.
    18. Alexander Eliseev & Sergei Seleznev, 2026. "Fake Date Tests: Can We Trust In-sample Accuracy of LLMs in Macroeconomic Forecasting?," Papers 2601.07992, arXiv.org, revised Mar 2026.
    19. Iñaki Aldasoro & Ajit Desai, 2025. "AI agents for cash management in payment systems," BIS Working Papers 1310, Bank for International Settlements.
    20. Fetzer, Thiemo & Lambert, Peter John & Feld, Bennet & Garg, Prashant, 2024. "AI-Generated Production Networks: Measurement and Applications to Global Trade," CAGE Online Working Paper Series 733, Competitive Advantage in the Global Economy (CAGE).
    21. Wang Xian & Chen Guomin & Varsha Arya & Kwok Tai Chui, 2024. "Examining the Influence of AI Chatbots on Semantic Web-Based Global Information Management in Various Industries," International Journal on Semantic Web and Information Systems (IJSWIS), IGI Global Scientific Publishing, vol. 20(1), pages 1-14, January.

    More about this item

    Keywords

    ;
    ;
    ;

    JEL classification:

    • A2 - General Economics and Teaching - - Economic Education and Teaching of Economics
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ireced:v:49:y:2025:i:c:s1477388025000131. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.journals.elsevier.com/international-review-of-economics-education .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.