IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/103934.html
   My bibliography  Save this paper

Experimental Evidence on Artificial Intelligence in the Classroom

Author

Listed:
  • Ferman, Bruno
  • Lima, Lycia
  • Riva, Flavio

Abstract

This paper investigates how technologies that use different combinations of artificial and human intelligence are incorporated into classroom instruction, and how they ultimately affect students' outcomes. We conducted a field experiment to study two technologies that allow teachers to outsource grading and feedback tasks on writing practices. The first technology is a fully automated evaluation system that provides instantaneous scores and feedback. The second one uses human graders as an additional resource to enhance grading and feedback quality in aspects in which the automated system arguably falls short. Both technologies significantly improved students' essay scores, and the additional inputs from human graders did not improve effectiveness. Furthermore, the technologies similarly helped teachers engage more frequently on nonroutine tasks that supported the individualization of pedagogy. Our results are informative about the potential of artificial intelligence to expand the set of tasks that can be automated, and on how advances in artificial intelligence may relocate human labor to tasks that remain out of reach of automation.

Suggested Citation

  • Ferman, Bruno & Lima, Lycia & Riva, Flavio, 2020. "Experimental Evidence on Artificial Intelligence in the Classroom," MPRA Paper 103934, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:103934
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/103934/1/MPRA_paper_103934.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:cdl:agrebk:qt15n8j26f is not listed on IDEAS
    2. Christopher J. Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2018. "More than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text-Messages," NBER Working Papers 24450, National Bureau of Economic Research, Inc.
    3. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    4. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    5. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    6. David H. Autor & Frank Levy & Richard J. Murnane, 2003. "The Skill Content of Recent Technological Change: An Empirical Exploration," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(4), pages 1279-1333.
    7. repec:cdl:ucscec:qt0rb5x6bf is not listed on IDEAS
    8. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    9. George Bulman & Robert W. Fairlie, 2015. "Technology and Education: Computers, Software, and the Internet," CESifo Working Paper Series 5570, CESifo.
    10. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    11. Bulman, G. & Fairlie, R.W., 2016. "Technology and Education," Handbook of the Economics of Education,, Elsevier.
    12. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    13. Acemoglu, Daron & Autor, David, 2011. "Skills, Tasks and Technologies: Implications for Employment and Earnings," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 12, pages 1043-1171, Elsevier.
    14. David H. Autor & Frank Levy & Richard J. Murnane, 2003. "The skill content of recent technological change: an empirical exploration," Proceedings, Federal Reserve Bank of San Francisco, issue nov.
    15. Ferman, Bruno & Ponczek, Vladimir, 2017. "Should we drop covariate cells with attrition problems?," MPRA Paper 80686, University Library of Munich, Germany.
    16. Benjamin N. York & Susanna Loeb, 2014. "One Step at a Time: The Effects of an Early Literacy Text Messaging Program for Parents of Preschoolers," NBER Working Papers 20659, National Bureau of Economic Research, Inc.
    17. Joel Mokyr & Chris Vickers & Nicolas L. Ziebarth, 2015. "The History of Technological Anxiety and the Future of Economic Growth: Is This Time Different?," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 31-50, Summer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ferman, Bruno & Lima, Lycia & Riva, Flávio, 2021. "Artificial Intelligence, Teacher Tasks and Individualized Pedagogy," SocArXiv qw249, Center for Open Science.
    2. repec:osf:socarx:qw249_v1 is not listed on IDEAS
    3. Falck, Oliver & Heimisch-Roecker, Alexandra & Wiederhold, Simon, 2021. "Returns to ICT skills," Research Policy, Elsevier, vol. 50(7).
    4. Díaz, Carlos & Dodel, Matías & Menese, Pablo, 2022. "Can one laptop per child reduce digital inequalities? ICT household access patterns under Plan Ceibal," Telecommunications Policy, Elsevier, vol. 46(9).
    5. Derksen, Laura & Michaud-Leclerc, Catherine & Souza, Pedro C.L., 2022. "Restricted access: How the internet can be used to promote reading and learning," Journal of Development Economics, Elsevier, vol. 155(C).
    6. Caselli, Mauro & Fracasso, Andrea & Scicchitano, Sergio & Traverso, Silvio & Tundis, Enrico, 2025. "What workers and robots do: An activity-based analysis of the impact of robotization on changes in local employment," Research Policy, Elsevier, vol. 54(1).
    7. Włoch, Renata & Śledziewska, Katarzyna & Rożynek, Satia, 2025. "Who's afraid of automation? Examining determinants of fear of automation in six European countries," Technology in Society, Elsevier, vol. 81(C).
    8. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers : The Impact of Student Access to Wikipedia," The Warwick Economics Research Paper Series (TWERPS) 1236, University of Warwick, Department of Economics.
    9. Per-Anders Edin & Peter Fredriksson & Martin Nybom & Björn Öckert, 2022. "The Rising Return to Noncognitive Skill," American Economic Journal: Applied Economics, American Economic Association, vol. 14(2), pages 78-100, April.
    10. Arntz, Melanie & Gregory, Terry & Zierahn-Weilage, Ulrich, 2019. "Digitalization and the Future of Work: Macroeconomic Consequences," IZA Discussion Papers 12428, Institute of Labor Economics (IZA).
    11. Daron Acemoglu & Pascual Restrepo, 2020. "The wrong kind of AI? Artificial intelligence and the future of labour demand," Cambridge Journal of Regions, Economy and Society, Cambridge Political Economy Society, vol. 13(1), pages 25-35.
    12. Fierro, Luca Eduardo & Caiani, Alessandro & Russo, Alberto, 2022. "Automation, Job Polarisation, and Structural Change," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 499-535.
    13. Gregory, Terry & Salomons, Anna & Zierahn, Ulrich, 2016. "Racing With or Against the Machine? Evidence from Europe," VfS Annual Conference 2016 (Augsburg): Demographic Change 145843, Verein für Socialpolitik / German Economic Association.
    14. Cortes, Guido Matias & Salvatori, Andrea, 2019. "Delving into the demand side: Changes in workplace specialization and job polarization," Labour Economics, Elsevier, vol. 57(C), pages 164-176.
    15. Van Overbeke, Toon, 2022. "(De)regulating automation: the rise of credit scoring and market-led banking in the UK and Germany," LSE Research Online Documents on Economics 114993, London School of Economics and Political Science, LSE Library.
    16. Eckhardt Bode & Stephan Brunow & Ingrid Ott & Alina Sorgner, 2019. "Worker Personality: Another Skill Bias beyond Education in the Digital Age," German Economic Review, Verein für Socialpolitik, vol. 20(4), pages 254-294, November.
    17. Nikolova, Milena & Cnossen, Femke & Nikolaev, Boris, 2024. "Robots, meaning, and self-determination," Research Policy, Elsevier, vol. 53(5).
    18. Maarten Goos & Melanie Arntz & Ulrich Zierahn & Terry Gregory & Stephanie Carretero Gomez & Ignacio Gonzalez Vazquez & Koen Jonkers, 2019. "The Impact of Technological Innovation on the Future of Work," JRC Working Papers on Labour, Education and Technology 2019-03, Joint Research Centre.
    19. Genz, Sabrina & Schnabel, Claus, 2021. "Digging into the digital divide: Workers' exposure to digitalization and its consequences for individual employment," Discussion Papers 118, Friedrich-Alexander University Erlangen-Nuremberg, Chair of Labour and Regional Economics.
    20. Naik, Gopal & Chitre, Chetan & Bhalla, Manaswini & Rajan, Jothsna, 2020. "Impact of use of technology on student learning outcomes: Evidence from a large-scale experiment in India," World Development, Elsevier, vol. 127(C).
    21. Rita Pető & Balázs Reizer, 2021. "Gender differences in the skill content of jobs," Journal of Population Economics, Springer;European Society for Population Economics, vol. 34(3), pages 825-864, July.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid
    • I25 - Health, Education, and Welfare - - Education - - - Education and Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:103934. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.