IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v6y2022i7d10.1038_s41562-022-01381-z.html
   My bibliography  Save this article

Experimental evidence on learning using low-tech when school is out

Author

Listed:
  • Noam Angrist

    (Youth Impact
    University of Oxford
    The World Bank)

  • Peter Bergman

    (Teachers College
    National Bureau of Economic Research (NBER))

  • Moitshepi Matsheng

    (Youth Impact
    Botswana National Youth Council)

Abstract

School closures occurred extensively during the COVID-19 pandemic, and occur in other settings, such as teacher strikes and natural disasters. The cost of school closures has proven to be substantial, particularly for households of lower socioeconomic status, but little evidence exists on how to mitigate these learning losses. This paper provides experimental evidence on strategies to support learning when schools close. We conduct a large-scale randomized trial testing two low-technology interventions—SMS messages and phone calls—with parents to support their child in Botswana. The combined treatment improves learning by 0.12 standard deviations, which translates to 0.89 standard deviations of learning per US$100, ranking among the most cost-effective interventions to improve learning. We develop remote assessment innovations, which show robust learning outcomes. Our findings have immediate policy relevance and long-run implications for the role of technology and parents to support education provision during school disruptions.

Suggested Citation

  • Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
  • Handle: RePEc:nat:nathum:v:6:y:2022:i:7:d:10.1038_s41562-022-01381-z
    DOI: 10.1038/s41562-022-01381-z
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-022-01381-z
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1038/s41562-022-01381-z?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Noam Angrist & Peter Bergman & David K. Evans & Susannah Hares & Matthew C. H. Jukes & Thato Letsomo, 2020. "Practical Lessons for Phone-Based Assessments of Learning," Working Papers 534, Center for Global Development, revised 10 Jul 2020.
    2. Abhijit Banerjee & Rukmini Banerji & James Berry & Esther Duflo & Harini Kannan & Shobhini Mukerji & Marc Shotland & Michael Walton, 2017. "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application," Journal of Economic Perspectives, American Economic Association, vol. 31(4), pages 73-102, Fall.
    3. Siebert, W. Stanley & Wei, Xiangdong & Wong, Ho Lun & Zhou, Xiang, 2018. "Student Feedback, Parent-Teacher Communication, and Academic Performance: Experimental Evidence from Rural China," IZA Discussion Papers 11347, Institute of Labor Economics (IZA).
    4. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1235-1264.
    5. Noam Angrist & Simeon Djankov & Pinelopi K. Goldberg & Harry A. Patrinos, 2021. "Measuring human capital using global learning data," Nature, Nature, vol. 592(7854), pages 403-408, April.
    6. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2021. "Telementoring and homeschooling during school closures: A randomized experiment in rural Bangladesh," Munich Papers in Political Economy 13, Munich School of Politics and Public Policy and the School of Management at the Technical University of Munich.
    7. David Jaume & Alexander Willén, 2019. "The Long-Run Effects of Teacher Strikes: Evidence from Argentina," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1097-1139.
    8. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    9. Gallego, Francisco A. & Malamud, Ofer & Pop-Eleches, Cristian, 2020. "Parental monitoring and children's internet use: The role of information, control, and cues," Journal of Public Economics, Elsevier, vol. 188(C).
    10. Uri Gneezy & John A. List & Jeffrey A. Livingston & Xiangdong Qin & Sally Sadoff & Yang Xu, 2019. "Measuring Success in Education: The Role of Effort on the Test Itself," American Economic Review: Insights, American Economic Association, vol. 1(3), pages 291-308, December.
    11. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
    12. Damien de Walque & Christine Valente, 2023. "Incentivizing School Attendance in the Presence of Parent-Child Information Frictions," American Economic Journal: Economic Policy, American Economic Association, vol. 15(3), pages 256-285, August.
    13. Kraft, Matthew A. & Rogers, Todd, 2015. "The underutilized potential of teacher-to-parent communication: Evidence from a field experiment," Economics of Education Review, Elsevier, vol. 47(C), pages 49-63.
    14. Muriel Niederle & Lise Vesterlund, 2007. "Do Women Shy Away From Competition? Do Men Compete Too Much?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(3), pages 1067-1101.
    15. Abhijit V. Banerjee & Rukmini Banerji & Esther Duflo & Rachel Glennerster & Stuti Khemani, 2010. "Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India," American Economic Journal: Economic Policy, American Economic Association, vol. 2(1), pages 1-30, February.
    16. Peter Bergman & Eric W. Chan, 2021. "Leveraging Parents through Low-Cost Technology: The Impact of High-Frequency Information on Student Achievement," Journal of Human Resources, University of Wisconsin Press, vol. 56(1), pages 125-158.
    17. Andre Nickow & Philip Oreopoulos & Vincent Quan, 2020. "The Impressive Effects of Tutoring on PreK-12 Learning: A Systematic Review and Meta-Analysis of the Experimental Evidence," NBER Working Papers 27476, National Bureau of Economic Research, Inc.
    18. Guilherme Lichand & Sharon Wolf, 2020. "Arm-wrestling in the classroom: the non-monotonic effects of monitoring teachers," ECON - Working Papers 357, Department of Economics - University of Zurich, revised Feb 2021.
    19. Anna Josephson & Talip Kilic & Jeffrey D. Michler, 2021. "Socioeconomic impacts of COVID-19 in low-income countries," Nature Human Behaviour, Nature, vol. 5(5), pages 557-565, May.
    20. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," CEPR Discussion Papers 15761, C.E.P.R. Discussion Papers.
    21. Peter Bergman, 2021. "Parent-Child Information Frictions and Human Capital Investment: Evidence from a Field Experiment," Journal of Political Economy, University of Chicago Press, vol. 129(1), pages 286-322.
    22. Samuel Berlinski & Matias Busso & Taryn Dinkelman & Claudia Martínez A., 2021. "Reducing Parent-School Information Gaps and Improving Education Outcomes: Evidence from High-Frequency Text Messages," NBER Working Papers 28581, National Bureau of Economic Research, Inc.
    23. Evans, David K. & Popova, Anna, 2016. "Cost-Effectiveness Analysis in Development: Accounting for Local Costs and Noisy Impacts," World Development, Elsevier, vol. 77(C), pages 262-276.
    24. Per Engzell & Arun Frey & Mark D. Verhagen, 2021. "Learning loss due to school closures during the COVID-19 pandemic," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(17), pages 2022376118-, April.
    25. Young, Alwyn, 2019. "Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results," LSE Research Online Documents on Economics 101401, London School of Economics and Political Science, LSE Library.
    26. Raj Chetty & John N. Friedman & Michael Stepner & The Opportunity Insights Team, 2020. "The Economic Impacts of COVID-19: Evidence from a New Public Database Built Using Private Sector Data," NBER Working Papers 27431, National Bureau of Economic Research, Inc.
    27. Bacher-Hicks, Andrew & Goodman, Joshua & Mulhern, Christine, 2021. "Inequality in household adaptation to schooling shocks: Covid-induced online learning engagement in real time," Journal of Public Economics, Elsevier, vol. 193(C).
    28. Zachary Parolin & Emma K. Lee, 2021. "Large socio-economic, geographic and demographic disparities exist in exposure to school closures," Nature Human Behaviour, Nature, vol. 5(4), pages 522-528, April.
    29. Jenny C. Aker & Rachid Boumnijel & Amanda McClelland & Niall Tierney, 2016. "Payment Mechanisms and Antipoverty Programs: Evidence from a Mobile Money Cash Transfer Experiment in Niger," Economic Development and Cultural Change, University of Chicago Press, vol. 65(1), pages 1-37.
    30. Noam Angrist & Peter Bergman & Caton Brewster & Moitshepi Matsheng, 2020. "Stemming Learning Loss During the Pandemic: A Rapid Randomized Trial of a Low-Tech Intervention in Botswana," CSAE Working Paper Series 2020-13, Centre for the Study of African Economies, University of Oxford.
    31. Eric Bettinger & Nina Cunha & Guilherme Lichand & Ricardo Madeira, 2020. "Are the effects of informational interventions driven by salience?," ECON - Working Papers 350, Department of Economics - University of Zurich, revised May 2021.
    32. Jenny C. Aker & Christopher Ksoll & Travis J. Lybbert, 2012. "Can Mobile Phones Improve Learning? Evidence from a Field Experiment in Niger," American Economic Journal: Applied Economics, American Economic Association, vol. 4(4), pages 94-120, October.
    33. Jenny C. Aker & Christopher Ksoll, 2020. "Can ABC Lead to Sustained 123? The Medium-Term Effects of a Technology-Enhanced Adult Education Program," Economic Development and Cultural Change, University of Chicago Press, vol. 68(3), pages 1081-1102.
    34. Uri Gneezy & Muriel Niederle & Aldo Rustichini, 2003. "Performance in Competitive Environments: Gender Differences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 1049-1074.
    35. Todd Rogers & Avi Feller, 2018. "Reducing student absences at scale by targeting parents’ misbeliefs," Nature Human Behaviour, Nature, vol. 2(5), pages 335-342, May.
    36. Christopher Doss & Erin M. Fahle & Susanna Loeb & Benjamin N. York, 2019. "More Than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text Messages," Journal of Human Resources, University of Wisconsin Press, vol. 54(3), pages 567-603.
    37. Gema Zamarro & Collin Hitt & Ildefonso Mendez, 2019. "When Students Don’t Care: Reexamining International Differences in Achievement and Student Effort," Journal of Human Capital, University of Chicago Press, vol. 13(4), pages 519-552.
    38. Alwyn Young, 2019. "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 557-598.
    39. João Pedro & Amer Hasan & Diana Goldemberg & Koen Geven & Syedah Aroob Iqbal, 2021. "Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates [Tackling Inequity in Education during and after COVID-19]," The World Bank Research Observer, World Bank, vol. 36(1), pages 1-40.
    40. Kalena E. Cortes & Hans Fricke & Susanna Loeb & David S. Song & Benjamin N. York, 2021. "Too Little or Too Much? Actionable Advice in an Early-Childhood Text Messaging Experiment," Education Finance and Policy, MIT Press, vol. 16(2), pages 209-232, Spring.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hardt, David & Nagler, Markus & Rincke, Johannes, 2023. "Tutoring in (online) higher education: Experimental evidence," Economics of Education Review, Elsevier, vol. 92(C).
    2. Abhijeet Singh & Mauricio Romero & Karthik Muralidharan, 2022. "Covid-19 Learning Loss and Recovery: Panel Data Evidence from India," NBER Working Papers 30552, National Bureau of Economic Research, Inc.
    3. Shi Jiao & Fang Liang, 2022. "Sustainable Development of High School English Learners in China: Motivation and Its Impact on Their English Achievement," Sustainability, MDPI, vol. 14(19), pages 1-19, October.
    4. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    5. Shi Jiao & Hui Jin & Zheng You & Jie Wang, 2022. "Motivation and Its Effect on Language Achievement: Sustainable Development of Chinese Middle School Students’ Second Language Learning," Sustainability, MDPI, vol. 14(16), pages 1-18, August.
    6. Wang, Liang Choon & Vlassopoulos, Michael & Islam, Asad & Hassan, Hashibul, 2023. "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh," IZA Discussion Papers 15920, Institute of Labor Economics (IZA).
    7. Hui Jin & Xu Ma & Shi Jiao, 2022. "Cultural Capital and Its Impact on Academic Achievement: Sustainable Development of Chinese High School Students," Sustainability, MDPI, vol. 14(22), pages 1-18, November.
    8. Jakubowski,Maciej Jan & Gajderowicz,Tomasz Janusz & Patrinos,Harry Anthony, 2024. "COVID-19, School Closures, and Student Learning Outcomes : New Global Evidence from PISA," Policy Research Working Paper Series 10666, The World Bank.
    9. Bastian A. Betthäuser & Anders M. Bach-Mortensen & Per Engzell, 2023. "A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic," Nature Human Behaviour, Nature, vol. 7(3), pages 375-385, March.
    10. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    11. Lucas Gortazar & Claudia Hupkau & Antonio Roldan, 2023. "Online tutoring works: experimental evidence from a program with vulnerable children," CEP Discussion Papers dp1908, Centre for Economic Performance, LSE.
    12. Rodriguez-Segura, Daniel & Schueler, Beth E., 2022. "Can learning be measured by phone? Evidence from Kenya," Economics of Education Review, Elsevier, vol. 90(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2020. "School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic," NBER Working Papers 28205, National Bureau of Economic Research, Inc.
    2. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2021. "Telementoring and homeschooling during school closures: A randomized experiment in rural Bangladesh," Munich Papers in Political Economy 13, Munich School of Politics and Public Policy and the School of Management at the Technical University of Munich.
    3. Esposito Acosta,Bruno Nicola & Sautmann,Anja, 2022. "Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya," Policy Research Working Paper Series 10098, The World Bank.
    4. Angrist, Noam & de Barros, Andreas & Bhula, Radhika & Chakera, Shiraz & Cummiskey, Chris & DeStefano, Joseph & Floretta, John & Kaffenberger, Michelle & Piper, Benjamin & Stern, Jonathan, 2021. "Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery," International Journal of Educational Development, Elsevier, vol. 84(C).
    5. Guido Neidhöfer & Nora Lustig & Mariano Tommasi, 2021. "Intergenerational transmission of lockdown consequences: prognosis of the longer-run persistence of COVID-19 in Latin America," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(3), pages 571-598, September.
    6. Werner, Katharina & Woessmann, Ludger, 2021. "The Legacy of COVID-19 in Education," IZA Discussion Papers 14796, Institute of Labor Economics (IZA).
    7. Beam, Emily A. & Mukherjee, Priya & Navarro-Sola, Laia, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," IZA Discussion Papers 15596, Institute of Labor Economics (IZA).
    8. Harry Anthony Patrinos, 2022. "Learning loss and learning recovery," DECISION: Official Journal of the Indian Institute of Management Calcutta, Springer;Indian Institute of Management Calcutta, vol. 49(2), pages 183-188, June.
    9. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    10. Boruchowicz, Cynthia & Parker, Susan W. & Robbins, Lindsay, 2022. "Time use of youth during a pandemic: Evidence from Mexico," World Development, Elsevier, vol. 149(C).
    11. Amaral, Sofia & Dinarte-Diaz, Lelys & Dominguez, Patricio & Perez-Vincent, Santiago M., 2024. "Helping families help themselves: The (Un)intended impacts of a digital parenting program," Journal of Development Economics, Elsevier, vol. 166(C).
    12. Rosa Sanchis-Guarner & José Montalbán & Felix Weinhardt, 2021. "Home Broadband and Human Capital Formation," CESifo Working Paper Series 8846, CESifo.
    13. Hermes, Henning & Lergetporer, Philipp & Peter, Frauke & Wiederhold, Simon, 2021. "Behavioral Barriers and the Socioeconomic Gap in Child Care Enrollment," IZA Discussion Papers 14698, Institute of Labor Economics (IZA).
    14. Henning Hermes & Philipp Lergetporer & Frauke Peter & Simon Wiederhold, 2021. "Application Barriers and the Socioeconomic Gap in Child Care Enrollment," CESifo Working Paper Series 9282, CESifo.
    15. Hikaru Kawarazaki & Minhaj Mahmud & Yasuyuki Sawada & Mai Seki, 2023. "Haste Makes No Waste: Positive Peer Effects of Classroom Speed Competition on Learning," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 85(4), pages 755-772, August.
    16. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    17. Mateo-Berganza Díaz, María Mercedes & Becerra, Laura & Hernández Agramonte, Juan Manuel & Lopez Boo, Florencia & Pérez Alfaro, Marcelo & Vasquez Echeverria, Alejandro, 2020. "Nudging Parents to Increase Preschool Attendance in Uruguay," IDB Publications (Working Papers) 10893, Inter-American Development Bank.
    18. Bastian A. Betthäuser & Anders M. Bach-Mortensen & Per Engzell, 2023. "A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic," Nature Human Behaviour, Nature, vol. 7(3), pages 375-385, March.
    19. Wang, Liang Choon & Vlassopoulos, Michael & Islam, Asad & Hassan, Hashibul, 2023. "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh," IZA Discussion Papers 15920, Institute of Labor Economics (IZA).
    20. Barrera-Osorio, Felipe & Gonzalez, Kathryn & Lagos, Francisco & Deming, David J., 2020. "Providing performance information in education: An experimental evaluation in Colombia," Journal of Public Economics, Elsevier, vol. 186(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:6:y:2022:i:7:d:10.1038_s41562-022-01381-z. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.