IDEAS home Printed from https://ideas.repec.org/p/wbk/wbrwps/11286.html

Nudging at Scale : Evidence from a Government Text Messaging Campaign during School Shutdowns in Punjab, Pakistan

Author

Listed:
  • Geven, Koen Martijn
  • Tahir, Ayesha
  • Fasih, Tazeen
  • Qureshi, Javaeria
  • Hasan, Amer
  • Fazili, Sheena
  • Malik, Rabea
  • Macdonald, Kevin Alan David

Abstract

Text and voice messages have emerged as a low-cost and popular tool for nudging recipients to change behavior. This paper presents findings from a randomized controlled trial designed to evaluate the impact of an information campaign using text and voice messages implemented in Punjab, Pakistan during the COVID-19-induced school closures. This campaign sought to increase study time and provide academic support while schools were closed and to encourage reenrollment when they opened, to reduce the number of dropouts. The campaign targeted girls enrolled in grades 5 to 7. Messages were sent out by a government institution, and the campaign lasted from October 2020 until November 2021, when schools had permanently re-opened. Households were randomized across three treatment groups and a control group that did not receive any messages. The first treatment group received gender-specific messages that explicitly referenced daughters in their households, and the second treatment group received gender-neutral messages. A third group was cross randomized across the first two treatment arms and received academic support messages (practice math problems and solutions). The results show that the messages increased reenrollment by 6.0 percentage points approximately three months after the intervention finished. Gender neutral messages (+8.9 percentage points) showed larger effect size on enrolment than gender-specific message s (+ 4.3 percentage points), although the difference is not statistically significant. The message program also increased learning outcomes by 0.2 standard deviation for Urdu and 0.2 standard deviation for math. The paper finds a small positive effect on the intensive margin of remote learning and an (equivalent) small negative effect on the intensive margin of outside tutoring. In line with similar studies on pandemic remediation efforts, the paper finds no effect of the academic support intervention on learning. The findings suggest that increased school enrollment played a role in supporting the observed increase in learning outcomes.

Suggested Citation

  • Geven, Koen Martijn & Tahir, Ayesha & Fasih, Tazeen & Qureshi, Javaeria & Hasan, Amer & Fazili, Sheena & Malik, Rabea & Macdonald, Kevin Alan David, 2026. "Nudging at Scale : Evidence from a Government Text Messaging Campaign during School Shutdowns in Punjab, Pakistan," Policy Research Working Paper Series 11286, The World Bank.
    • Handle: RePEc:wbk:wbrwps:11286
    as

    Download full text from publisher

    File URL: https://documents.worldbank.org/curated/en/099958301082615003/pdf/IDU-784bac57-5b74-4494-98d9-c21a3dcc18bd.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Noam Angrist & Peter Bergman & David K. Evans & Susannah Hares & Matthew C. H. Jukes & Thato Letsomo, 2020. "Practical Lessons for Phone-Based Assessments of Learning," Working Papers 534, Center for Global Development, revised 10 Jul 2020.
    2. Leonardo Bursztyn & Alessandra L. González & David Yanagizawa-Drott, 2020. "Misperceived Social Norms: Women Working Outside the Home in Saudi Arabia," American Economic Review, American Economic Association, vol. 110(10), pages 2997-3029, October.
    3. S. Nageeb Ali & Roland Bénabou, 2020. "Image versus Information: Changing Societal Norms and Optimal Privacy," American Economic Journal: Microeconomics, American Economic Association, vol. 12(3), pages 116-164, August.
    4. Omar Al-Ubaydli & John A. List & Dana Suskind, 2019. "The Science of Using Science: Towards an Understanding of the Threats to Scaling Experiments," NBER Working Papers 25848, National Bureau of Economic Research, Inc.
    5. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    6. Laura Moscoviz & David K. Evans, 2022. "Learning Loss and Student Dropouts during the COVID-19 Pandemic: A Review of the Evidence Two Years after Schools Shut Down," Working Papers 609, Center for Global Development.
    7. Kidman, Rachel & Breton, Etienne & Behrman, Jere & Kohler, Hans-Peter, 2022. "Returning to school after COVID-19 closures: Who is missing in Malawi?," International Journal of Educational Development, Elsevier, vol. 93(C).
    8. Maya Escueta & Andre Joshua Nickow & Philip Oreopoulos & Vincent Quan, 2020. "Upgrading Education with Technology: Insights from Experimental Research," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 897-996, December.
    9. Damien de Walque & Christine Valente, 2023. "Incentivizing School Attendance in the Presence of Parent-Child Information Frictions," American Economic Journal: Economic Policy, American Economic Association, vol. 15(3), pages 256-285, August.
    10. Bastian A. Betthäuser & Anders M. Bach-Mortensen & Per Engzell, 2023. "A systematic review and meta-analysis of the evidence on learning during the COVID-19 pandemic," Nature Human Behaviour, Nature, vol. 7(3), pages 375-385, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dang, Hai-Anh H. & Oseni, Gbemisola & Abanokova, Kseniya, 2025. "Educational inequalities during COVID-19: Results from longitudinal surveys in Sub-Saharan Africa," International Journal of Educational Development, Elsevier, vol. 112(C).
    2. Singh, Abhijeet & Romero, Mauricio & Muralidharan, Karthik, 2024. "COVID-19 Learning loss and recovery," University of California at San Diego, Economics Working Paper Series qt3jj1b8hb, Department of Economics, UC San Diego.
    3. Neranjan Senarath & Wathsala Dedunupitiya & Gayanthi Ilangarathna & Isuru Thilakasiri & Bhagya Samarakoon & Lakshitha Ramanayake & Roshan Godaliyadda & Parakrama Ekanayake & Vijitha Herath & Mallika P, 2024. "Family income does not necessarily lead to a holistic e-learning experience during COVID-19: a study in Sri Lanka," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 11(1), pages 1-14, December.
    4. Ingela Alger & Sergey Gavrilets & Patrick Durkee, 2024. "Proximate and ultimate drivers of norms and norm change," Post-Print hal-04730213, HAL.
    5. Fernández-Duque, Mauricio, 2022. "The probability of pluralistic ignorance," Journal of Economic Theory, Elsevier, vol. 202(C).
    6. Lafortune, Jeanne & Pugatch, Todd & Tessada, José & Ubfal, Diego, 2024. "Can gamified online training make high school students more entrepreneurial? Experimental evidence from Rwanda," Economics of Education Review, Elsevier, vol. 101(C).
    7. Rachel Cassidy & Anaya Dam & Wendy Janssens & Umair Kiani & Karlijn Morsink, 2022. "Father of the bride, or steel magnolias? Targeting men, women or both to reduce child marriage," IFS Working Papers W22/50, Institute for Fiscal Studies.
    8. Asadullah, M. Niaz, 2024. "Back to school after COVID-19 pandemic: Resumption or transitional disruption?," International Journal of Educational Development, Elsevier, vol. 109(C).
    9. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    10. Paul Romer & Roberto Samaniego & Remi Jedwab & Asif M. Islam, 2025. "Scars of pandemics from lost schooling and experience: aggregate implications and gender differences through the lens of COVID-19," Journal of Economic Growth, Springer, vol. 30(1), pages 1-47, March.
    11. Prudencio, Daniel & Balmori-de-la-Miyar, Jose & Silverio-Murillo, Adan & Sobrino, Fernanda, 2024. "Examining COVID-19’s disruptive effect on education in Mexican universities," International Journal of Educational Development, Elsevier, vol. 111(C).
    12. Angrist, Noam & Bergman, Peter & Matsheng, Moitshepi, 2020. "School's Out: Experimental Evidence on Limiting Learning Loss Using," IZA Discussion Papers 14009, IZA Network @ LISER.
    13. Rodriguez-Segura, Daniel & Schueler, Beth E., 2022. "Can learning be measured by phone? Evidence from Kenya," Economics of Education Review, Elsevier, vol. 90(C).
    14. Alicia Brandt & Hamid R. Oskorouchi & Alfonso Sousa-Poza, 2024. "The effect of SMS nudges on higher education performance," Empirical Economics, Springer, vol. 66(5), pages 2311-2334, May.
    15. Gortazar, Lucas & Hupkau, Claudia & Roldán-Monés, Antonio, 2024. "Online tutoring works: Experimental evidence from a program with vulnerable children," Journal of Public Economics, Elsevier, vol. 232(C).
    16. Yang, Liu & Zhang, Lei, 2024. "Online teaching, gender differences and education outcomes: Evidence from Chinese urban high schools during the COVID-19," Journal of Comparative Economics, Elsevier, vol. 52(2), pages 534-553.
    17. Gambi, Letizia & De Witte, Kristof, 2024. "The resilience in test scores and the mitigating effect of summer schools one year after the COVID-19 school closures," Labour Economics, Elsevier, vol. 90(C).
    18. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2020. "School’s Out: Experimental Evidence on Limiting Learning Loss Using “Low-Tech” in a Pandemic," NBER Working Papers 28205, National Bureau of Economic Research, Inc.
    19. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    20. Susanti, Dewi & Anand, Gautam & Arifin, Firda Arianti, 2025. "Leveraging school principals to address learning loss in Indonesia through group and individual targeting," International Journal of Educational Development, Elsevier, vol. 112(C).

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wbk:wbrwps:11286. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Roula I. Yazigi (email available below). General contact details of provider: https://edirc.repec.org/data/dvewbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.