IDEAS home Printed from https://ideas.repec.org/p/zbw/i4rdps/241.html
   My bibliography  Save this paper

A Comment on "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh"

Author

Listed:
  • Fiala, Lenka
  • Fitzgerald, Jack
  • Kujansuu, Essi
  • Mikola, Derek
  • Valenta, David
  • Aparicio, Juan P.
  • Wiebe, Michael
  • Webb, Matthew D.
  • Brodeur, Abel

Abstract

Wang et al. (2024) report that Bangladeshi students randomly given access to lessons on a phone server saw significant learning gains during COVID- 19 school closures. We identify three sets of anomalies. First, this experiment shares participants with another experiment conducted simultaneously in the same region, but test scores for the same children systematically differ between the two experiments. Second, test scores for treated participants exhibit a uniform upward shift that is completely insensitive to the number of lessons children complete. Third, numerous documentation inconsistencies (e.g., concerning survey materials, randomization procedures, etc.) cast doubt on the study's data.

Suggested Citation

  • Fiala, Lenka & Fitzgerald, Jack & Kujansuu, Essi & Mikola, Derek & Valenta, David & Aparicio, Juan P. & Wiebe, Michael & Webb, Matthew D. & Brodeur, Abel, 2025. "A Comment on "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh"," I4R Discussion Paper Series 241, The Institute for Replication (I4R).
    • Handle: RePEc:zbw:i4rdps:241
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/321368/1/I4R-DP241.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Aparicio, Juan P. & Cook, Nikolai & Mikola, Derek & Rogeberg, Ole & Valenta, David & Wiebe, Michael & Bonander, Carl & Brodeur, Abel, 2025. "Comment on "Telementoring and Homeschooling during School Closures: A Randomized Experiment in Rural Bangladesh" by Hassan et al," I4R Discussion Paper Series 240, The Institute for Replication (I4R).
    2. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    3. Gortazar, Lucas & Hupkau, Claudia & Roldán-Monés, Antonio, 2024. "Online tutoring works: Experimental evidence from a program with vulnerable children," Journal of Public Economics, Elsevier, vol. 232(C).
    4. David Rhys Bernard & Gharad Bryan & Sylvain Chabé-Ferret & Jonathan de Quidt & Jasmin Claire Fliegner & Roland Rathelot, 2023. "How Much Should We Trust Observational Estimates? Accumulating Evidence Using Randomized Controlled Trials with Imperfect Compliance," Working Papers 976, Queen Mary University of London, School of Economics and Finance.
    5. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    6. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2025. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," The Review of Economics and Statistics, MIT Press, vol. 107(3), pages 589-604, May.
    7. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," CEPR Discussion Papers 15761, C.E.P.R. Discussion Papers.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aparicio, Juan P. & Cook, Nikolai & Mikola, Derek & Rogeberg, Ole & Valenta, David & Wiebe, Michael & Bonander, Carl & Brodeur, Abel, 2025. "Comment on "Telementoring and Homeschooling during School Closures: A Randomized Experiment in Rural Bangladesh" by Hassan et al," I4R Discussion Paper Series 240, The Institute for Replication (I4R).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jakubowski,Maciej Jan & Gajderowicz,Tomasz Janusz & Patrinos,Harry Anthony, 2024. "COVID-19, School Closures, and Student Learning Outcomes : New Global Evidence from PISA," Policy Research Working Paper Series 10666, The World Bank.
    2. Aparicio, Juan P. & Cook, Nikolai & Mikola, Derek & Rogeberg, Ole & Valenta, David & Wiebe, Michael & Bonander, Carl & Brodeur, Abel, 2025. "Comment on "Telementoring and Homeschooling during School Closures: A Randomized Experiment in Rural Bangladesh" by Hassan et al," I4R Discussion Paper Series 240, The Institute for Replication (I4R).
    3. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2024. "Telementoring and Homeschooling During School Closures: a Randomised Experiment in Rural Bangladesh," The Economic Journal, Royal Economic Society, vol. 134(662), pages 2418-2438.
    4. Liang Choon Wang & Michael Vlassopoulos & Asad Islam & Hashibul Hassan, 2024. "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 562-601.
    5. Gajderowicz, Tomasz & Jakubowski, Maciej & Kennedy, Alec & Christrup Kjeldsen, Christian & Patrinos, Harry Anthony & Strietholt, Rolf, 2024. "The Learning Crisis: Three Years After Covid-19," GLO Discussion Paper Series 1543, Global Labor Organization (GLO).
    6. Beam, Emily A. & Mukherjee, Priya & Navarro-Sola, Laia, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," IZA Discussion Papers 15596, Institute of Labor Economics (IZA).
    7. Cristia, Julian P. & Cueto, Santiago & Malamud, Ofer & Aulagnon, Raphaëlle, 2024. "Streaking to Success: The Effects of Highlighting Streaks on Student Effort and Achievement," IDB Publications (Working Papers) 13498, Inter-American Development Bank.
    8. Hardt, David & Nagler, Markus & Rincke, Johannes, 2023. "Tutoring in (online) higher education: Experimental evidence," Economics of Education Review, Elsevier, vol. 92(C).
    9. Alicia Brandt & Hamid R. Oskorouchi & Alfonso Sousa-Poza, 2024. "The effect of SMS nudges on higher education performance," Empirical Economics, Springer, vol. 66(5), pages 2311-2334, May.
    10. Abhijeet Singh & Mauricio Romero & Karthik Muralidharan, 2022. "Covid-19 Learning Loss and Recovery: Panel Data Evidence from India," CESifo Working Paper Series 10031, CESifo.
    11. Gortazar, Lucas & Hupkau, Claudia & Roldán-Monés, Antonio, 2024. "Online tutoring works: Experimental evidence from a program with vulnerable children," Journal of Public Economics, Elsevier, vol. 232(C).
    12. Elisabetta Aurino & Sharon Wolf, 2024. "A ‘smart buy' for all? Unequal and unintended consequences of a messaging program for child education," UB School of Economics Working Papers 2024/461, University of Barcelona School of Economics.
    13. Lucas Gortazar & Claudia Hupkau & Antonio Roldan, 2023. "Online tutoring works: experimental evidence from a program with vulnerable children," CEP Discussion Papers dp1908, Centre for Economic Performance, LSE.
    14. Dang, Hai-Anh H. & Oseni, Gbemisola & Abanokova, Kseniya, 2025. "Educational inequalities during COVID-19: Results from longitudinal surveys in Sub-Saharan Africa," International Journal of Educational Development, Elsevier, vol. 112(C).
    15. Philipp Ketz & Adam McCloskey, 2025. "Short and Simple Confidence Intervals When the Directions of Some Effects are Known," The Review of Economics and Statistics, MIT Press, vol. 107(3), pages 820-834, May.
    16. Ballestar, María Teresa & Mir, Miguel Cuerdo & Pedrera, Luis Miguel Doncel & Sainz, Jorge, 2024. "Effectiveness of tutoring at school: A machine learning evaluation," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    17. Jorge Luis García & James J. Heckman, 2021. "Early childhood education and life‐cycle health," Health Economics, John Wiley & Sons, Ltd., vol. 30(S1), pages 119-141, November.
    18. repec:cdl:ucsdec:qt3jj1b8hb is not listed on IDEAS
    19. Derksen, Laura & Kerwin, Jason Theodore & Reynoso, Natalia Ordaz & Sterck, Olivier, 2021. "Appointments: A More Effective Commitment Device for Health Behaviors," SocArXiv y8gh7, Center for Open Science.
    20. Josep Amer-Mestre & Alaitz Ayarza-Astigarraga & Marta C. Lopes, 2024. "E-learning engagement gap during school closures: differences by academic performance," Applied Economics, Taylor & Francis Journals, vol. 56(3), pages 337-359, January.
    21. Lafortune, Jeanne & Pugatch, Todd & Tessada, José & Ubfal, Diego, 2022. "Can interactive online training make high school students more entrepreneurial? Experimental evidence from Rwanda," GLO Discussion Paper Series 1041, Global Labor Organization (GLO).

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:i4rdps:241. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://www.i4replication.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.