IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/103934.html

Experimental Evidence on Artificial Intelligence in the Classroom

Author

Listed:
  • Ferman, Bruno
  • Lima, Lycia
  • Riva, Flavio

Abstract

This paper investigates how technologies that use different combinations of artificial and human intelligence are incorporated into classroom instruction, and how they ultimately affect students' outcomes. We conducted a field experiment to study two technologies that allow teachers to outsource grading and feedback tasks on writing practices. The first technology is a fully automated evaluation system that provides instantaneous scores and feedback. The second one uses human graders as an additional resource to enhance grading and feedback quality in aspects in which the automated system arguably falls short. Both technologies significantly improved students' essay scores, and the additional inputs from human graders did not improve effectiveness. Furthermore, the technologies similarly helped teachers engage more frequently on nonroutine tasks that supported the individualization of pedagogy. Our results are informative about the potential of artificial intelligence to expand the set of tasks that can be automated, and on how advances in artificial intelligence may relocate human labor to tasks that remain out of reach of automation.

Suggested Citation

  • Ferman, Bruno & Lima, Lycia & Riva, Flavio, 2020. "Experimental Evidence on Artificial Intelligence in the Classroom," MPRA Paper 103934, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:103934
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/103934/1/MPRA_paper_103934.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    2. Christopher J. Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2018. "More than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text-Messages," NBER Working Papers 24450, National Bureau of Economic Research, Inc.
    3. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    4. David H. Autor & Frank Levy & Richard J. Murnane, 2003. "The skill content of recent technological change: an empirical exploration," Proceedings, Federal Reserve Bank of San Francisco, issue Nov.
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    7. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    8. George Bulman & Robert W. Fairlie, 2015. "Technology and Education: Computers, Software, and the Internet," CESifo Working Paper Series 5570, CESifo.
    9. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    10. Ferman, Bruno & Ponczek, Vladimir, 2017. "Should we drop covariate cells with attrition problems?," MPRA Paper 80686, University Library of Munich, Germany.
    11. Acemoglu, Daron & Autor, David, 2011. "Skills, Tasks and Technologies: Implications for Employment and Earnings," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 12, pages 1043-1171, Elsevier.
    12. Bulman, G. & Fairlie, R.W., 2016. "Technology and Education," Handbook of the Economics of Education,, Elsevier.
    13. Benjamin N. York & Susanna Loeb, 2014. "One Step at a Time: The Effects of an Early Literacy Text Messaging Program for Parents of Preschoolers," NBER Working Papers 20659, National Bureau of Economic Research, Inc.
    14. Joel Mokyr & Chris Vickers & Nicolas L. Ziebarth, 2015. "The History of Technological Anxiety and the Future of Economic Growth: Is This Time Different?," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 31-50, Summer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ferman, Bruno & Lima, Lycia & Riva, Flávio, 2021. "Artificial Intelligence, Teacher Tasks and Individualized Pedagogy," SocArXiv qw249, Center for Open Science.
    2. repec:osf:socarx:qw249_v1 is not listed on IDEAS
    3. Derksen, Laura & Michaud-Leclerc, Catherine & Souza, Pedro C.L., 2022. "Restricted access: How the internet can be used to promote reading and learning," Journal of Development Economics, Elsevier, vol. 155(C).
    4. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers : The Impact of Student Access to Wikipedia," The Warwick Economics Research Paper Series (TWERPS) 1236, University of Warwick, Department of Economics.
    5. Naik, Gopal & Chitre, Chetan & Bhalla, Manaswini & Rajan, Jothsna, 2020. "Impact of use of technology on student learning outcomes: Evidence from a large-scale experiment in India," World Development, Elsevier, vol. 127(C).
    6. Perera, Marcelo & Aboal, Diego, 2019. "The impact of a mathematics computer-assisted learning platform on students' mathematics test scores," MERIT Working Papers 2019-007, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    7. Díaz, Carlos & Dodel, Matías & Menese, Pablo, 2022. "Can one laptop per child reduce digital inequalities? ICT household access patterns under Plan Ceibal," Telecommunications Policy, Elsevier, vol. 46(9).
    8. So Yoon Ahn & Youjin Hahn & Semee Yoon, 2021. "Can New Learning Opportunities Reshape Gender Attitudes for Girls?: Field Evidence from Tanzania," Working Papers 2021-046, Human Capital and Economic Opportunity Working Group.
    9. Falck, Oliver & Heimisch-Roecker, Alexandra & Wiederhold, Simon, 2021. "Returns to ICT skills," Research Policy, Elsevier, vol. 50(7).
    10. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers: The Impact of Student Access to Wikipedia," CAGE Online Working Paper Series 450, Competitive Advantage in the Global Economy (CAGE).
    11. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    12. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    13. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    14. Caselli, Mauro & Fracasso, Andrea & Scicchitano, Sergio & Traverso, Silvio & Tundis, Enrico, 2025. "What workers and robots do: An activity-based analysis of the impact of robotization on changes in local employment," Research Policy, Elsevier, vol. 54(1).
    15. Battisti & M. Del Gatto & A. Gravina & C. Parmeter, 2021. "Robots versus labor skills: a complementarity/substitutability analysis," Working Paper CRENoS 202104, Centre for North South Economic Research, University of Cagliari and Sassari, Sardinia.
    16. Islam, Asad & Kwon, Sungoh & Masood, Eema & Prakash, Nishith & Sabarwal, Shwetlena & Saraswat, Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," GLO Discussion Paper Series 526, Global Labor Organization (GLO).
    17. Włoch, Renata & Śledziewska, Katarzyna & Rożynek, Satia, 2025. "Who's afraid of automation? Examining determinants of fear of automation in six European countries," Technology in Society, Elsevier, vol. 81(C).
    18. Ali, Ayesha & Qazi, Ihsan Ayyub, 2023. "Countering misinformation on social media through educational interventions: Evidence from a randomized experiment in Pakistan," Journal of Development Economics, Elsevier, vol. 163(C).
    19. Raquel Sebastian, 2018. "Explaining job polarisation in Spain from a task perspective," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 9(2), pages 215-248, June.
    20. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    21. Ruchir Agarwal & Patrick Gaule, 2026. "Developing Math Talent Worldwide: Evidence from a Global RCT," Bristol Economics Discussion Papers 26/846, School of Economics, University of Bristol, UK.

    More about this item

    Keywords

    ;
    ;
    ;
    ;

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid
    • I25 - Health, Education, and Welfare - - Education - - - Education and Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:103934. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.