IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp18513.html

Guidance Over Adoption: Experimental Evidence on AI-Assisted Learning

Author

Listed:
  • Gallegos, Sebastian

    (Universidad Adolfo Ibañez)

Abstract

This paper estimates the causal effect of a large language model-based study assistant on student behavior and learning outcomes in a natural field setting with real academic stakes. I design and deploy a course-specific AI assistant (GPT-UAI) for undergraduate econometrics and evaluate it through two randomized interventions implemented across seven coordinated course sections at a selective university in Chile. The first intervention targets the extensive margin of use, encouraging GPT-UAI adoption prior to the midterm exam. The encouragement raises the GPT's awareness and reported usage, but does not change its perceived value and does not improve midterm performance. The second intervention targets use at the intensive margin, providing guidance on learning-oriented usage for the final exam. Guidance shifts interactions with GPT-UAI toward tutor-style engagement, increases perceived usefulness by 0.38 standard deviations, improves final-exam performance by 0.21 standard deviations, and raises the probability of earning a passing exam grade by 12 percentage points. The findings suggest that learning gains arise less from adoption than from guiding how students use course-specific AI assistants.

Suggested Citation

  • Gallegos, Sebastian, 2026. "Guidance Over Adoption: Experimental Evidence on AI-Assisted Learning," IZA Discussion Papers 18513, IZA Network @ LISER.
    • Handle: RePEc:iza:izadps:dp18513
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp18513.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fischer, Mira & Rau, Holger A. & Rilke, Rainer Michael, 2025. "AI tutoring enhances student learning without crowding out reading effort," Discussion Papers, Research Unit: Market Behavior SP II 2025-203, WZB Berlin Social Science Center.
    2. Keshav Agrawal & Susan Athey & Ayush Kanodia & Shanjukta Nath & Emil Palikot, 2026. "The Economics of Algorithmic Personalization: Evidence from an Educational Technology Platform," NBER Working Papers 34950, National Bureau of Economic Research, Inc.
    3. Cueto, Santiago & Beuermann, Diether W. & Cristia, Julian & Malamud, Ofer & Pardo, Francisco, 2025. "Laptops in the long run: Evidence from the one laptop per child program in rural Peru," Journal of Public Economics, Elsevier, vol. 252(C).
    4. Alberto Abadie & Susan Athey & Guido W Imbens & Jeffrey M Wooldridge, 2023. "When Should You Adjust Standard Errors for Clustering?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 138(1), pages 1-35.
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Julian Cristia & Pablo Ibarrarán & Santiago Cueto & Ana Santiago & Eugenio Severín, 2017. "Technology and Child Development: Evidence from the One Laptop per Child Program," American Economic Journal: Applied Economics, American Economic Association, vol. 9(3), pages 295-320, July.
    7. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    8. Fischer, Mira & Rau, Holger A. & Rilke, Rainer Michael, 2025. "AI Tutoring Enhances Student Learning Without Crowding Out Reading Effort," IZA Discussion Papers 18338, IZA Network @ LISER.
    9. Mira Fischer & Holger A. Rau & Rainer Michael Rilke, 2025. "AI Tutoring Enhances Student Learning Without Crowding Out Reading Effort," Rationality and Competition Discussion Paper Series 557, CRC TRR 190 Rationality and Competition.
    10. Ofer Malamud & Cristian Pop-Eleches, 2011. "Home Computer Use and the Development of Human Capital," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(2), pages 987-1027.
    11. Joseph P. Romano & Michael Wolf, 2005. "Stepwise Multiple Testing as Formalized Data Snooping," Econometrica, Econometric Society, vol. 73(4), pages 1237-1282, July.
    12. Miriam Bruhn & David McKenzie, 2009. "In Pursuit of Balance: Randomization in Practice in Development Field Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
    13. Edward Vytlacil, 2002. "Independence, Monotonicity, and Latent Index Models: An Equivalence Result," Econometrica, Econometric Society, vol. 70(1), pages 331-341, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers : The Impact of Student Access to Wikipedia," The Warwick Economics Research Paper Series (TWERPS) 1236, University of Warwick, Department of Economics.
    2. Derksen, Laura & Leclerc, Catherine Michaud & Souza, Pedro CL, 2019. "Searching for Answers: The Impact of Student Access to Wikipedia," CAGE Online Working Paper Series 450, Competitive Advantage in the Global Economy (CAGE).
    3. Derksen, Laura & Michaud-Leclerc, Catherine & Souza, Pedro C.L., 2022. "Restricted access: How the internet can be used to promote reading and learning," Journal of Development Economics, Elsevier, vol. 155(C).
    4. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    5. Cardim, Joana & Molina-Millán, Teresa & Vicente, Pedro C., 2023. "Can technology improve the classroom experience in primary education? An African experiment on a worldwide program," Journal of Development Economics, Elsevier, vol. 164(C).
    6. Malamud, Ofer & Cueto, Santiago & Cristia, Julian & Beuermann, Diether W., 2019. "Do children benefit from internet access? Experimental evidence from Peru," Journal of Development Economics, Elsevier, vol. 138(C), pages 41-56.
    7. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    8. Sanchis-Guarner, Rosa & Montalbán, José & Weinhardt, Felix, 2025. "Home broadband and human capital formation," Economics of Education Review, Elsevier, vol. 108(C).
    9. Akabayashi, Hideo & Ruberg, Tim & Shikishima, Chizuru & Yamashita, Jun, 2023. "Education-oriented and care-oriented preschools: Implications on child development," Labour Economics, Elsevier, vol. 84(C).
    10. Yanguas, Maria Lucia, 2020. "Technology and educational choices: Evidence from a one-laptop-per-child program," Economics of Education Review, Elsevier, vol. 76(C).
    11. Kramer, Berber & Pattnaik, Subhransu & Ward, Patrick S. & Xu, Yingchen, 2024. "Impacts of an innovative credit + insurance bundle for marginalized farmers: Evidence from a cluster randomized trial in Odisha, India," IFPRI discussion papers 2288, International Food Policy Research Institute (IFPRI).
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Naik, Gopal & Chitre, Chetan & Bhalla, Manaswini & Rajan, Jothsna, 2020. "Impact of use of technology on student learning outcomes: Evidence from a large-scale experiment in India," World Development, Elsevier, vol. 127(C).
    14. David J. Deming, 2022. "Four Facts about Human Capital," Journal of Economic Perspectives, American Economic Association, vol. 36(3), pages 75-102, Summer.
    15. Hong, Junjie & Liu, Wanlin & Zhang, Qing, 2024. "Closing the digital divide: The impact of teachers’ ICT use on student achievement in China," Journal of Comparative Economics, Elsevier, vol. 52(3), pages 697-713.
    16. Han, Sukjin & Yang, Shenshen, 2024. "A computational approach to identification of treatment effects for policy evaluation," Journal of Econometrics, Elsevier, vol. 240(1).
    17. Yue Ma & Robert Fairlie & Prashant Loyalka & Scott Rozelle, 2024. "Isolating the “Tech” from EdTech: Experimental Evidence on Computer-Assisted Learning in China," Economic Development and Cultural Change, University of Chicago Press, vol. 72(4), pages 1923-1962.
    18. Araya, Roberto & Arias Ortiz, Elena & Bottan, Nicolas & Cristia, Julian, 2025. "Integrating learning platforms within regular school time: experimental evidence from Chilean primary schools," Economics of Education Review, Elsevier, vol. 106(C).
    19. Prashant Poddar & Valentina Rotondi & Ridhi Kashyap, 2025. "Access to technology and foundational math proficiency among students: empirical evidence from India," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-15, December.
    20. Beuermann, Diether & Cristia, Julian P. & Cruz-Aguayo, Yyannu & Cueto, Santiago & Malamud, Ofer, 2012. "Home Computers and Child Outcomes: Short-Term Impacts from a Randomized Experiment in Peru," IDB Publications (Working Papers) 4204, Inter-American Development Bank.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • O33 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Technological Change: Choices and Consequences; Diffusion Processes
    • D83 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Search; Learning; Information and Knowledge; Communication; Belief; Unawareness

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp18513. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Mark Fallak (email available below). General contact details of provider: https://edirc.repec.org/data/izaaalu.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.