IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/103934.html
   My bibliography  Save this paper

Experimental Evidence on Artificial Intelligence in the Classroom

Author

Listed:
  • Ferman, Bruno
  • Lima, Lycia
  • Riva, Flavio

Abstract

This paper investigates how technologies that use different combinations of artificial and human intelligence are incorporated into classroom instruction, and how they ultimately affect students' outcomes. We conducted a field experiment to study two technologies that allow teachers to outsource grading and feedback tasks on writing practices. The first technology is a fully automated evaluation system that provides instantaneous scores and feedback. The second one uses human graders as an additional resource to enhance grading and feedback quality in aspects in which the automated system arguably falls short. Both technologies significantly improved students' essay scores, and the additional inputs from human graders did not improve effectiveness. Furthermore, the technologies similarly helped teachers engage more frequently on nonroutine tasks that supported the individualization of pedagogy. Our results are informative about the potential of artificial intelligence to expand the set of tasks that can be automated, and on how advances in artificial intelligence may relocate human labor to tasks that remain out of reach of automation.

Suggested Citation

  • Ferman, Bruno & Lima, Lycia & Riva, Flavio, 2020. "Experimental Evidence on Artificial Intelligence in the Classroom," MPRA Paper 103934, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:103934
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/103934/1/MPRA_paper_103934.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Anderson, Michael L, 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Department of Agricultural & Resource Economics, UC Berkeley, Working Paper Series qt15n8j26f, Department of Agricultural & Resource Economics, UC Berkeley.
    2. Christopher J. Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2018. "More than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text-Messages," NBER Working Papers 24450, National Bureau of Economic Research, Inc.
    3. Ferman, Bruno & Finamor, Lucas & Lima, Lycia, 2019. "Are Public Schools Ready to Integrate Math Classes with Khan Academy?," MPRA Paper 94736, University Library of Munich, Germany.
    4. George Bulman & Robert W. Fairlie, 2015. "Technology and Education: Computers, Software, and the Internet," CESifo Working Paper Series 5570, CESifo.
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    7. David H. Autor & Frank Levy & Richard J. Murnane, 2003. "The skill content of recent technological change: an empirical exploration," Proceedings, Federal Reserve Bank of San Francisco, issue Nov.
    8. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    9. Anderson, Michael L., 2008. "Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects," Journal of the American Statistical Association, American Statistical Association, vol. 103(484), pages 1481-1495.
    10. Bulman, G. & Fairlie, R.W., 2016. "Technology and Education," Handbook of the Economics of Education,, Elsevier.
    11. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    12. Acemoglu, Daron & Autor, David, 2011. "Skills, Tasks and Technologies: Implications for Employment and Earnings," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 12, pages 1043-1171, Elsevier.
    13. Ferman, Bruno & Ponczek, Vladimir, 2017. "Should we drop covariate cells with attrition problems?," MPRA Paper 80686, University Library of Munich, Germany.
    14. Benjamin N. York & Susanna Loeb, 2014. "One Step at a Time: The Effects of an Early Literacy Text Messaging Program for Parents of Preschoolers," NBER Working Papers 20659, National Bureau of Economic Research, Inc.
    15. Joel Mokyr & Chris Vickers & Nicolas L. Ziebarth, 2015. "The History of Technological Anxiety and the Future of Economic Growth: Is This Time Different?," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 31-50, Summer.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ferman, Bruno & Lima, Lycia & Riva, Flávio, 2021. "Artificial Intelligence, Teacher Tasks and Individualized Pedagogy," SocArXiv qw249, Center for Open Science.
    2. Derksen, Laura & Michaud-Leclerc, Catherine & Souza, Pedro C.L., 2022. "Restricted access: How the internet can be used to promote reading and learning," Journal of Development Economics, Elsevier, vol. 155(C).
    3. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers : The Impact of Student Access to Wikipedia," The Warwick Economics Research Paper Series (TWERPS) 1236, University of Warwick, Department of Economics.
    4. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers: The Impact of Student Access to Wikipedia," CAGE Online Working Paper Series 450, Competitive Advantage in the Global Economy (CAGE).
    5. So Yoon Ahn & Youjin Hahn & Semee Yoon, 2021. "Can New Learning Opportunities Reshape Gender Attitudes for Girls?: Field Evidence from Tanzania," Working Papers 2021-046, Human Capital and Economic Opportunity Working Group.
    6. Islam, Asad & Kwon, Sungoh & Masood, Eema & Prakash, Nishith & Sabarwal, Shwetlena & Saraswat, Deepak, 2020. "When Goal-Setting Forges Ahead but Stops Short," GLO Discussion Paper Series 526, Global Labor Organization (GLO).
    7. Johnston, Jamie & Ksoll, Christopher, 2022. "Effectiveness of interactive satellite-transmitted instruction: Experimental evidence from Ghanaian primary schools," Economics of Education Review, Elsevier, vol. 91(C).
    8. Falck, Oliver & Heimisch-Roecker, Alexandra & Wiederhold, Simon, 2021. "Returns to ICT skills," Research Policy, Elsevier, vol. 50(7).
    9. Fairlie Robert W., 2016. "Do Boys and Girls Use Computers Differently, and Does It Contribute to Why Boys do Worse in School Than Girls?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 16(1), pages 59-96, January.
    10. Francisco Gallego & Emma Näslund-Hadley & Mariana Alfonso, 2017. "Tailoring Instruction to Improve Mathematics Skills in Preschools: A Randomized Evaluation," Documentos de Trabajo 487, Instituto de Economia. Pontificia Universidad Católica de Chile..
    11. Naik, Gopal & Chitre, Chetan & Bhalla, Manaswini & Rajan, Jothsna, 2020. "Impact of use of technology on student learning outcomes: Evidence from a large-scale experiment in India," World Development, Elsevier, vol. 127(C).
    12. Díaz, Carlos & Dodel, Matías & Menese, Pablo, 2022. "Can one laptop per child reduce digital inequalities? ICT household access patterns under Plan Ceibal," Telecommunications Policy, Elsevier, vol. 46(9).
    13. Rita Pető & Balázs Reizer, 2021. "Gender differences in the skill content of jobs," Journal of Population Economics, Springer;European Society for Population Economics, vol. 34(3), pages 825-864, July.
    14. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    15. Lisa Simon, 2019. "Microeconometric Analyses on Determinants of Individual Labour Market Outcomes," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 83.
    16. Ali, Ayesha & Qazi, Ihsan Ayyub, 2023. "Countering misinformation on social media through educational interventions: Evidence from a randomized experiment in Pakistan," Journal of Development Economics, Elsevier, vol. 163(C).
    17. Islam, Asad & Lee, Wang-Sheng & Nicholas, Aaron, 2021. "The Effects of Chess Instruction on Academic and Non-cognitive Outcomes: Field Experimental Evidence from a Developing Country," Journal of Development Economics, Elsevier, vol. 150(C).
    18. Perera, Marcelo & Aboal, Diego, 2019. "The impact of a mathematics computer-assisted learning platform on students' mathematics test scores," MERIT Working Papers 2019-007, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    19. Cilliers, Jacobus & Fleisch, Brahm & Kotze, Janeli & Mohohlwane, Nompumelelo & Taylor, Stephen & Thulare, Tsegofatso, 2022. "Can virtual replace in-person coaching? Experimental evidence on teacher professional development and student learning," Journal of Development Economics, Elsevier, vol. 155(C).
    20. Sonia Bhalotra & Martin Karlsson & Therese Nilsson & Nina Schwarz, 2022. "Infant Health, Cognitive Performance, and Earnings: Evidence from Inception of the Welfare State in Sweden," The Review of Economics and Statistics, MIT Press, vol. 104(6), pages 1138-1156, November.

    More about this item

    Keywords

    artificial intelligence; technological change; automated writing evaluation; routine and nonroutine tasks;
    All these keywords.

    JEL classification:

    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I22 - Health, Education, and Welfare - - Education - - - Educational Finance; Financial Aid
    • I25 - Health, Education, and Welfare - - Education - - - Education and Economic Development

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:103934. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.