IDEAS home Printed from https://ideas.repec.org/a/nat/nathum/v7y2023i3d10.1038_s41562-022-01506-4.html
   My bibliography  Save this article

A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic

Author

Listed:
  • Bastian A. Betthäuser

    (Centre for Research on Social Inequalities (CRIS), Sciences Po
    University of Oxford
    University of Oxford)

  • Anders M. Bach-Mortensen

    (University of Oxford)

  • Per Engzell

    (University of Oxford
    University College London
    Stockholm University)

Abstract

To what extent has the learning progress of school-aged children slowed down during the COVID-19 pandemic? A growing number of studies address this question, but findings vary depending on context. Here we conduct a pre-registered systematic review, quality appraisal and meta-analysis of 42 studies across 15 countries to assess the magnitude of learning deficits during the pandemic. We find a substantial overall learning deficit (Cohen’s d = −0.14, 95% confidence interval −0.17 to −0.10), which arose early in the pandemic and persists over time. Learning deficits are particularly large among children from low socio-economic backgrounds. They are also larger in maths than in reading and in middle-income countries relative to high-income countries. There is a lack of evidence on learning progress during the pandemic in low-income countries. Future research should address this evidence gap and avoid the common risks of bias that we identify.

Suggested Citation

  • Bastian A. Betthäuser & Anders M. Bach-Mortensen & Per Engzell, 2023. "A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic," Nature Human Behaviour, Nature, vol. 7(3), pages 375-385, March.
  • Handle: RePEc:nat:nathum:v:7:y:2023:i:3:d:10.1038_s41562-022-01506-4
    DOI: 10.1038/s41562-022-01506-4
    as

    Download full text from publisher

    File URL: https://www.nature.com/articles/s41562-022-01506-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1038/s41562-022-01506-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Aucejo, Esteban M. & Romano, Teresa Foy, 2016. "Assessing the effect of school days and absences on test score performance," Economics of Education Review, Elsevier, vol. 55(C), pages 70-87.
    2. Contini Dalit & Di Tommaso Maria Laura & Muratori Caterina & Piazzalunga Daniela & Schiavon Lucia, 2022. "Who Lost the Most? Mathematics Achievement during the COVID-19 Pandemic," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 22(2), pages 399-408, April.
    3. Noam Angrist & Simeon Djankov & Pinelopi K. Goldberg & Harry A. Patrinos, 2021. "Measuring human capital using global learning data," Nature, Nature, vol. 592(7854), pages 403-408, April.
    4. David Jaume & Alexander Willén, 2019. "The Long-Run Effects of Teacher Strikes: Evidence from Argentina," Journal of Labor Economics, University of Chicago Press, vol. 37(4), pages 1097-1139.
    5. Andrea Ichino & Rudolf Winter-Ebmer, 2004. "The Long-Run Educational Cost of World War II," Journal of Labor Economics, University of Chicago Press, vol. 22(1), pages 57-86, January.
    6. Michèle Belot & Dinand Webbink, 2010. "Do Teacher Strikes Harm Educational Attainment of Students?," LABOUR, CEIS, vol. 24(4), pages 391-406, December.
    7. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    8. Lee Elliot Major & Andrew Eyles & Stephen Machin, 2021. "Learning loss since lockdown: variation across the home nations," CEP Covid-19 Analyses cepcovid-19-023, Centre for Economic Performance, LSE.
    9. Nicola Fuchs-Schünde & Dirk Krueger & Alexander Ludwig & Irina Popova, 2022. "The Long-Term Distributional and Welfare Effects of Covid-19 School Closures," The Economic Journal, Royal Economic Society, vol. 132(645), pages 1647-1683.
    10. Per Engzell & Arun Frey & Mark D. Verhagen, 2021. "Learning loss due to school closures during the COVID-19 pandemic," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(17), pages 2022376118-, April.
    11. Guilherme Lichand & Carlos Alberto Doria & Onicio Leal-Neto & João Paulo Cossi Fernandes, 2022. "Publisher Correction: The impacts of remote learning in secondary education during the pandemic in Brazil," Nature Human Behaviour, Nature, vol. 6(8), pages 1180-1180, August.
    12. Patrinos,Harry Anthony & Vegas,Emiliana & Carter-Rau,Rohan, 2022. "An Analysis of COVID-19 Student Learning Loss," Policy Research Working Paper Series 10033, The World Bank.
    13. Guilherme Lichand & Carlos Alberto Doria & Onicio Leal-Neto & João Paulo Cossi Fernandes, 2022. "The impacts of remote learning in secondary education during the pandemic in Brazil," Nature Human Behaviour, Nature, vol. 6(8), pages 1079-1086, August.
    14. Dalton Conley & Tim Johnson, 2021. "Opinion: Past is future for the era of COVID-19 research in the social sciences," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 118(13), pages 2104155118-, March.
    15. Zachary Parolin & Emma K. Lee, 2021. "Large socio-economic, geographic and demographic disparities exist in exposure to school closures," Nature Human Behaviour, Nature, vol. 5(4), pages 522-528, April.
    16. Joshua Goodman, 2014. "Flaking Out: Student Absences and Snow Days as Disruptions of Instructional Time," NBER Working Papers 20221, National Bureau of Economic Research, Inc.
    17. Lee Elliot Major & Andrew Eyles & Stephen Machin, 2021. "Learning loss since lockdown," CentrePiece - The magazine for economic performance 614, Centre for Economic Performance, LSE.
    18. Kaffenberger, Michelle, 2021. "Modelling the long-run learning impact of the Covid-19 learning shock: Actions to (more than) mitigate loss," International Journal of Educational Development, Elsevier, vol. 81(C).
    19. Hevia, Felipe J. & Vergara-Lope, Samana & Velásquez-Durán, Anabel & Calderón, David, 2022. "Estimation of the fundamental learning loss and learning poverty related to COVID-19 pandemic in Mexico," International Journal of Educational Development, Elsevier, vol. 88(C).
    20. Ardington, Cally & Wills, Gabrielle & Kotze, Janeli, 2021. "COVID-19 learning losses: Early grade reading in South Africa," International Journal of Educational Development, Elsevier, vol. 86(C).
    21. Nicola Bazoli & Sonia Marzadro & Antonio Schizzerotto & Loris Vergolini, 2022. "Learning Loss and Students’ Social Origins During the Covid-19 Pandemic in Italy," FBK-IRVAPP Working Papers 2022-03, Research Institute for the Evaluation of Public Policies (IRVAPP), Bruno Kessler Foundation.
    22. João Pedro & Amer Hasan & Diana Goldemberg & Koen Geven & Syedah Aroob Iqbal, 2021. "Simulating the Potential Impacts of COVID-19 School Closures on Schooling and Learning Outcomes: A Set of Global Estimates [Tackling Inequity in Education during and after COVID-19]," The World Bank Research Observer, World Bank, vol. 36(1), pages 1-40.
    23. Angrist, Noam & de Barros, Andreas & Bhula, Radhika & Chakera, Shiraz & Cummiskey, Chris & DeStefano, Joseph & Floretta, John & Kaffenberger, Michelle & Piper, Benjamin & Stern, Jonathan, 2021. "Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery," International Journal of Educational Development, Elsevier, vol. 84(C).
    24. Sébastien Goudeau & Camille Sanrey & Arnaud Stanczak & Antony Manstead & Céline Darnon, 2021. "Why lockdown and distance learning during the COVID-19 pandemic are likely to increase the social class achievement gap," Nature Human Behaviour, Nature, vol. 5(10), pages 1273-1281, October.
    25. Andreu Arenas & Lucas Gortazar, 2022. "Learning loss one year after school closures: Evidence from the Basque Country," Working Papers 2022/03, Institut d'Economia de Barcelona (IEB).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Alderighi, Lorenzo & Ballatore, Rosario M. & Tonello, Marco, 2023. "Hidden drop-out: Secondary education (unseen) failure in pandemic times," GLO Discussion Paper Series 1293, Global Labor Organization (GLO).
    2. Nalini Prasad & Christian Gillitzer, 2023. "The Effect of School Closures on Standardized Test Scores: Evidence from a Zero-COVID Environment," Discussion Papers 2023-13, School of Economics, The University of New South Wales.
    3. Dong Yang & Peng Chen & Kai Wang & Zhuoran Li & Chen Zhang & Ronghuai Huang, 2023. "Parental Involvement and Student Engagement: A Review of the Literature," Sustainability, MDPI, vol. 15(7), pages 1-17, March.
    4. Contreras, M. Ignacia & Duryea, Suzanne & Martínez A., Claudia, 2023. "The effect of the pandemic on the transition to tertiary education in Chile: A focus on students with disabilities," International Journal of Educational Development, Elsevier, vol. 100(C).
    5. Breidenbach, Philipp & Hörnig, Lukas & Schaffner, Sandra, 2023. "The divergence of school track choices after Covid-19," Ruhr Economic Papers 1032, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Kuzmanic, Danilo & Valenzuela, Juan Pablo & Claro, Susana & Canales, Andrea & Cerda, Daniela & Undurraga, Eduardo A., 2023. "Socioeconomic disparities in the reopening of schools during the pandemic in Chile," International Journal of Educational Development, Elsevier, vol. 100(C).
    7. Lucas Gortazar & Claudia Hupkau & Antonio Roldan, 2023. "Online tutoring works: experimental evidence from a program with vulnerable children," CEP Discussion Papers dp1908, Centre for Economic Performance, LSE.
    8. Asadullah, M. Niaz & Tham, Eric, 2023. "Learning and happiness during Covid-19 school closure in urban Malaysia," International Journal of Educational Development, Elsevier, vol. 101(C).
    9. Léonard Moulin & Mara Soncin, 2023. "Persistent and Gender-Unequal Impacts of the COVID-19 Pandemic on Student Outcomes in Italy," Working Papers 277, French Institute for Demographic Studies.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ardington, Cally & Wills, Gabrielle & Kotze, Janeli, 2021. "COVID-19 learning losses: Early grade reading in South Africa," International Journal of Educational Development, Elsevier, vol. 86(C).
    2. ASAKAWA Shinsuke & OHTAKE Fumio & SANO Shinpei, 2023. "The Impact of the COVID-19 Pandemic on the Academic Achievement of Elementary and Junior High School Students: Analysis using administrative data from Amagasaki City," Discussion papers 23066, Research Institute of Economy, Trade and Industry (RIETI).
    3. Kuzmanic, Danilo & Valenzuela, Juan Pablo & Claro, Susana & Canales, Andrea & Cerda, Daniela & Undurraga, Eduardo A., 2023. "Socioeconomic disparities in the reopening of schools during the pandemic in Chile," International Journal of Educational Development, Elsevier, vol. 100(C).
    4. Monroy-Gómez-Franco, Luis & Vélez-Grajales, Roberto & López-Calva, Luis F., 2022. "The potential effects of the COVID-19 pandemic on learnings," International Journal of Educational Development, Elsevier, vol. 91(C).
    5. Mustafa Uğraş & Erdal Zengin & Stamatis Papadakis & Michail Kalogiannakis, 2023. "Early Childhood Learning Losses during COVID-19: Systematic Review," Sustainability, MDPI, vol. 15(7), pages 1-29, April.
    6. Guido Neidhöfer & Nora Lustig & Mariano Tommasi, 2021. "Intergenerational transmission of lockdown consequences: prognosis of the longer-run persistence of COVID-19 in Latin America," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(3), pages 571-598, September.
    7. Marín Llanes, Lucas & Rodríguez Pico, Mariana & Maldonado, Darío & García, Sandra, 2023. "Learning inequality during Covid-19: Evidence from secondary schools in Colombia," International Journal of Educational Development, Elsevier, vol. 100(C).
    8. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    9. Jo Blanden & Matthias Doepke & Jan Stuhler, 2022. "Education inequality," CEP Discussion Papers dp1849, Centre for Economic Performance, LSE.
    10. Oikawa, Masato & Tanaka, Ryuichi & Bessho, Shun-ichiro & Kawamura, Akira & Noguchi, Haruko, 2022. "Do Class Closures Affect Students' Achievements? Heterogeneous Effects of Students' Socioeconomic Backgrounds," IZA Discussion Papers 15751, Institute of Labor Economics (IZA).
    11. Shinsuke Asakawa & Fumio Ohtake, 2021. "Impact of Temporary School Closure Due to COVID-19 on the Academic Achievement of Elementary School Students," Discussion Papers in Economics and Business 21-14, Osaka University, Graduate School of Economics.
    12. Neidhöfer, Guido & Lustig, Nora & Larroulet, Patricio, 2022. "Nowcasting the impact of COVID-19 on education, intergenerational mobility and earnings inequality in Sub-Saharan Africa," ZEW Discussion Papers 22-022, ZEW - Leibniz Centre for European Economic Research.
    13. Nicola Fuchs-Schündeln & Dirk Krueger & André Kurmann & Etienne Lalé & Alexander Ludwig & Irina Popova, 2023. "The Fiscal and Welfare Effects of Policy Responses to the Covid-19 School Closures," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(1), pages 35-98, March.
    14. Heidmann, Laure & Neirac, Lucie & Andreu, Sandra & Conceiçao, Pierre & Eteve, Yann & Fabre, Marianne & Vourc'h, Ronan, 2023. "Delayed learning to read and write during the COVID-19 pandemic: longitudinal study of the heterogeneous effects on all first graders in France," SocArXiv qn9a8, Center for Open Science.
    15. Werner, Katharina & Woessmann, Ludger, 2021. "The Legacy of COVID-19 in Education," IZA Discussion Papers 14796, Institute of Labor Economics (IZA).
    16. Beam, Emily A. & Mukherjee, Priya & Navarro-Sola, Laia, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," IZA Discussion Papers 15596, Institute of Labor Economics (IZA).
    17. Monroy-Gómez-Franco, Luis Angel, 2021. "Modelling the learning impacts of educational disruptions in the short and long run," SocArXiv z6x5s, Center for Open Science.
    18. Kamila Cygan-Rehm, 2022. "Lifetime Consequences of Lost Instructional Time in the Classroom: Evidence from Shortened School Years," CESifo Working Paper Series 9892, CESifo.
    19. Harry Anthony Patrinos, 2022. "Learning loss and learning recovery," DECISION: Official Journal of the Indian Institute of Management Calcutta, Springer;Indian Institute of Management Calcutta, vol. 49(2), pages 183-188, June.
    20. Asadullah, M. Niaz & Tham, Eric, 2023. "Learning and happiness during Covid-19 school closure in urban Malaysia," International Journal of Educational Development, Elsevier, vol. 101(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nat:nathum:v:7:y:2023:i:3:d:10.1038_s41562-022-01506-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.nature.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.