IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/11125.html
   My bibliography  Save this paper

From Chalkboards to Chatbots : Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria

Author

Listed:
  • Martin Elias De Simone
  • Federico Hernan Tiberti
  • Maria Rebeca Barron Rodriguez
  • Federico Alfredo Manolio
  • Wuraola Mosuro
  • Eliot Jolomi Dikoru

Abstract

This study evaluates the impact of a program leveraging large language models for virtual tutoring in secondary education in Nigeria. Using a randomized controlled trial, the program deployed Microsoft Copilot (powered by GPT-4) to support first-year senior secondary students in English language learning over six weeks. The intervention demonstrated a significant improvement of 0.31 standard deviation on an assessment that included English topics aligned with the Nigerian curriculum, knowledge of artificial intelligence and digital skills. The effect on English, the main outcome of interest, was of 0.23 standard deviations. Cost-effectiveness analysis revealed substantial learning gains, equating to 1.5 to 2 years of ’business-as-usual’ schooling, situating the intervention among some of the most cost-effective programs to improve learning outcomes. An analysis of heterogeneous effects shows that while the program benefits students across the baseline ability distribution, the largest effects are for female students, and those with higher initial academic performance. The findings highlight that artificial intelligence-powered tutoring, when designed and used properly, can have transformative impacts in the education sector in low-resource settings.

Suggested Citation

  • Martin Elias De Simone & Federico Hernan Tiberti & Maria Rebeca Barron Rodriguez & Federico Alfredo Manolio & Wuraola Mosuro & Eliot Jolomi Dikoru, 2025. "From Chalkboards to Chatbots : Evaluating the Impact of Generative AI on Learning Outcomes in Nigeria," Policy Research Working Paper Series 11125, The World Bank.
    • Handle: RePEc:wbk:wbrwps:11125
    as

    Download full text from publisher

    File URL: https://documents.worldbank.org/curated/en/099548105192529324/pdf/IDU-c09f40d8-9ff8-42dc-b315-591157499be7.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1235-1264.
    2. Di Mo & Linxiu Zhang & Renfu Luo & Qinghe Qu & Weiming Huang & Jiafu Wang & Yajie Qiao & Matthew Boswell & Scott Rozelle, 2014. "Integrating computer-assisted learning into a regular curriculum: evidence from a randomised experiment in rural schools in Shaanxi," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 300-323, September.
    3. Jonathan Guryan & Jens Ludwig & Monica P. Bhatt & Philip J. Cook & Jonathan M. V. Davis & Kenneth Dodge & George Farkas & Roland G. Fryer Jr. & Susan Mayer & Harold Pollack & Laurence Steinberg & Greg, 2023. "Not Too Late: Improving Academic Outcomes among Adolescents," American Economic Review, American Economic Association, vol. 113(3), pages 738-765, March.
    4. Angel-Urdinola,Diego & Avitabile,Ciro & Chinen,Marjorie H, 2023. "Can Digital Personalized Learning for Mathematics Remediation Level the Playing Field inHigher Education ? Experimental Evidence from Ecuador," Policy Research Working Paper Series 10483, The World Bank.
    5. Michael Gerlich, 2025. "AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking," Societies, MDPI, vol. 15(1), pages 1-28, January.
    6. Daniel Rodriguez-Segura, 2022. "EdTech in Developing Countries: A Review of the Evidence [From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application]," The World Bank Research Observer, World Bank, vol. 37(2), pages 171-203.
    7. Angrist, Noam & Evans, David K. & Filmer, Deon & Glennerster, Rachel & Rogers, Halsey & Sabarwal, Shwetlena, 2025. "How to improve education outcomes most efficiently? A review of the evidence using a unified metric," Journal of Development Economics, Elsevier, vol. 172(C).
    8. David S. Lee, 2005. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," NBER Working Papers 11721, National Bureau of Economic Research, Inc.
    9. Konstantin Büchel & Martina Jakob & Christoph Kühnhanss & Daniel Steffen & Aymo Brunetti, 2022. "The Relative Effectiveness of Teachers and Learning Software: Evidence from a Field Experiment in El Salvador," Journal of Labor Economics, University of Chicago Press, vol. 40(3), pages 737-777.
    10. Hirotake Ito & Keiko Kasai & Hiromu Nishiuchi & Makiko Nakamuro, 2021. "Does Computer-Aided Instruction Improve Children's Cognitive and Noncognitive Skills?," Asian Development Review, MIT Press, vol. 38(1), pages 98-118, March.
    11. Angrist, Noam & Aurino, Elisabetta & Patrinos, Harry A. & Psacharopoulos, George & Vegas, Emiliana & Nordjo, Ralph & Wong, Brad, 2023. "Improving Learning in Low- and Lower-Middle-Income Countries," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 14(S1), pages 55-80, June.
    12. Roland G. Fryer Jr. & Meghan Howard-Noveck, 2020. "High-Dosage Tutoring and Reading Achievement: Evidence from New York City," Journal of Labor Economics, University of Chicago Press, vol. 38(2), pages 421-452.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Verónica Cabezas & José Ignacio Cuesta & Francisco Gallego, 2021. "Does Short-Term School Tutoring have Medium-Term Effects? Experimental Evidence from Chile," Documentos de Trabajo 565, Instituto de Economia. Pontificia Universidad Católica de Chile..
    2. Cueto, Santiago & Beuermann, Diether & Cristia, Julian P. & Malamud, Ofer & Pardo, Francisco, 2024. "Laptops in the Long-Run: Evidence from the One Laptop per Child Program in Rural Peru," IDB Publications (Working Papers) 13786, Inter-American Development Bank.
    3. Cardim, Joana & Molina-Millán, Teresa & Vicente, Pedro C., 2023. "Can technology improve the classroom experience in primary education? An African experiment on a worldwide program," Journal of Development Economics, Elsevier, vol. 164(C).
    4. Eric Bettinger & Robert Fairlie & Anastasia Kapuza & Elena Kardanova & Prashant Loyalka & Andrey Zakharov, 2023. "Diminishing Marginal Returns to Computer‐Assisted Learning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(2), pages 552-570, March.
    5. George Bulman & Robert W. Fairlie, 2015. "Technology and Education: Computers, Software, and the Internet," CESifo Working Paper Series 5570, CESifo.
    6. Pedro S. Martins, 2017. "(How) Do Non-Cognitive Skills Programs Improve Adolescent School Achievement? Experimental Evidence," Working Papers 81, Queen Mary, University of London, School of Business and Management, Centre for Globalisation Research.
    7. Lafortune, Jeanne & Pugatch, Todd & Tessada, José & Ubfal, Diego, 2024. "Can gamified online training make high school students more entrepreneurial? Experimental evidence from Rwanda," Economics of Education Review, Elsevier, vol. 101(C).
    8. Facundo Albornoz & María Victoria Anauati & Melina Furman & Mariana Luzuriaga & María Eugenia Podestá & Inés Taylor, 2020. "Training to Teach Science: Experimental Evidence from Argentina," The World Bank Economic Review, World Bank, vol. 34(2), pages 393-417.
    9. Hojun Lee & Youngsik Kim, 2023. "Exploring the Effects of Computer and Smart Device-Assisted Learning on Students’ Achievements: Empirical Evidence from Korea," Sustainability, MDPI, vol. 15(17), pages 1-16, September.
    10. Lars J. Kirkebøen & Trude Gunnes & Lena Lindenskov & Marte Rønning, 2021. "Didactic methods and small-group instruction for low-performing adolescents in mathematics. Results from a randomized controlled trial," Discussion Papers 957, Statistics Norway, Research Department.
    11. Naik, Gopal & Chitre, Chetan & Bhalla, Manaswini & Rajan, Jothsna, 2020. "Impact of use of technology on student learning outcomes: Evidence from a large-scale experiment in India," World Development, Elsevier, vol. 127(C).
    12. Hong, Junjie & Liu, Wanlin & Zhang, Qing, 2024. "Closing the digital divide: The impact of teachers’ ICT use on student achievement in China," Journal of Comparative Economics, Elsevier, vol. 52(3), pages 697-713.
    13. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    14. Maria Zumbuehl & Stefanie Hof & Stefan C. Wolter, 2020. "Private tutoring and academic achievement in a selective education system," Economics of Education Working Paper Series 0169, University of Zurich, Department of Business Administration (IBW), revised Oct 2022.
    15. Yue Ma & Robert Fairlie & Prashant Loyalka & Scott Rozelle, 2024. "Isolating the “Tech” from EdTech: Experimental Evidence on Computer-Assisted Learning in China," Economic Development and Cultural Change, University of Chicago Press, vol. 72(4), pages 1923-1962.
    16. Bonesrønning, Hans & Finseraas, Henning & Hardoy, Ines & Iversen, Jon Marius Vaag & Nyhus, Ole Henning & Opheim, Vibeke & Salvanes, Kari Vea & Sandsør, Astrid Marie Jorde & Schøne, Pål, 2022. "Small-group instruction to improve student performance in mathematics in early grades: Results from a randomized field experiment," Journal of Public Economics, Elsevier, vol. 216(C).
    17. Bin Tang & Te-Tien Ting & Chyi-In Wu & Yue Ma & Di Mo & Wei-Ting Hung & Scott Rozelle, 2020. "The Impact of Online Computer Assisted Learning at Home for Disadvantaged Children in Taiwan: Evidence from a Randomized Experiment," Sustainability, MDPI, vol. 12(23), pages 1-16, December.
    18. Angrist, Noam & Bergman, Peter & Matsheng, Moitshepi, 2020. "School's Out: Experimental Evidence on Limiting Learning Loss Using," IZA Discussion Papers 14009, Institute of Labor Economics (IZA).
    19. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2020. "School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic," NBER Working Papers 28205, National Bureau of Economic Research, Inc.
    20. Makiko NAKAMURO & Hirotake ITO, 2020. "The Effect of Computer Assisted Learning on Children's Cognitive and Noncognitive Skills: Evidence from a Randomized Experiment in Cambodia," Discussion papers 20074, Research Institute of Economy, Trade and Industry (RIETI).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:11125. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.