IDEAS home Printed from https://ideas.repec.org/p/zbw/i4rdps/241.html

A Comment on "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh"

Author

Listed:
  • Fiala, Lenka
  • Fitzgerald, Jack
  • Kujansuu, Essi
  • Mikola, Derek
  • Valenta, David
  • Aparicio, Juan P.
  • Wiebe, Michael
  • Webb, Matthew D.
  • Brodeur, Abel

Abstract

Wang et al. (2024) report that Bangladeshi students randomly given access to lessons on a phone server saw significant learning gains during COVID- 19 school closures. We identify three sets of anomalies. First, this experiment shares participants with another experiment conducted simultaneously in the same region, but test scores for the same children systematically differ between the two experiments. Second, test scores for treated participants exhibit a uniform upward shift that is completely insensitive to the number of lessons children complete. Third, numerous documentation inconsistencies (e.g., concerning survey materials, randomization procedures, etc.) cast doubt on the study's data.

Suggested Citation

  • Fiala, Lenka & Fitzgerald, Jack & Kujansuu, Essi & Mikola, Derek & Valenta, David & Aparicio, Juan P. & Wiebe, Michael & Webb, Matthew D. & Brodeur, Abel, 2025. "A Comment on "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh"," I4R Discussion Paper Series 241, The Institute for Replication (I4R).
    • Handle: RePEc:zbw:i4rdps:241
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/321368/1/I4R-DP241.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Aparicio, Juan P. & Cook, Nikolai & Mikola, Derek & Rogeberg, Ole & Valenta, David & Wiebe, Michael & Bonander, Carl & Brodeur, Abel, 2025. "Comment on "Telementoring and Homeschooling during School Closures: A Randomized Experiment in Rural Bangladesh" by Hassan et al," I4R Discussion Paper Series 240, The Institute for Replication (I4R).
    2. Crawfurd, Lee & Evans, David K. & Hares, Susannah & Sandefur, Justin, 2023. "Live tutoring calls did not improve learning during the COVID-19 pandemic in Sierra Leone," Journal of Development Economics, Elsevier, vol. 164(C).
    3. Gortazar, Lucas & Hupkau, Claudia & Roldán-Monés, Antonio, 2024. "Online tutoring works: Experimental evidence from a program with vulnerable children," Journal of Public Economics, Elsevier, vol. 232(C).
    4. David Rhys Bernard & Gharad Bryan & Sylvain Chabé-Ferret & Jonathan de Quidt & Jasmin Claire Fliegner & Roland Rathelot, 2023. "How Much Should We Trust Observational Estimates? Accumulating Evidence Using Randomized Controlled Trials with Imperfect Compliance," Working Papers 976, Queen Mary University of London, School of Economics and Finance.
    5. Noam Angrist & Peter Bergman & Moitshepi Matsheng, 2022. "Experimental evidence on learning using low-tech when school is out," Nature Human Behaviour, Nature, vol. 6(7), pages 941-950, July.
    6. Carlana, Michela & La Ferrara, Eliana, 2021. "Apart but Connected: Online Tutoring and Student Outcomes during the COVID-19 Pandemic," CEPR Discussion Papers 15761, C.E.P.R. Discussion Papers.
    7. Karthik Muralidharan & Mauricio Romero & Kaspar Wüthrich, 2025. "Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments," The Review of Economics and Statistics, MIT Press, vol. 107(3), pages 589-604, May.
    8. Matthew A. Kraft & John A. List & Jeffrey A. Livingston & Sally Sadoff, 2022. "Online Tutoring by College Volunteers: Experimental Evidence from a Pilot Program," AEA Papers and Proceedings, American Economic Association, vol. 112, pages 614-618, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aparicio, Juan P. & Cook, Nikolai & Mikola, Derek & Rogeberg, Ole & Valenta, David & Wiebe, Michael & Bonander, Carl & Brodeur, Abel, 2025. "Comment on "Telementoring and Homeschooling during School Closures: A Randomized Experiment in Rural Bangladesh" by Hassan et al," I4R Discussion Paper Series 240, The Institute for Replication (I4R).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gortazar, Lucas & Hupkau, Claudia & Roldán-Monés, Antonio, 2024. "Online tutoring works: Experimental evidence from a program with vulnerable children," Journal of Public Economics, Elsevier, vol. 232(C).
    2. Liang Choon Wang & Michael Vlassopoulos & Asad Islam & Hashibul Hassan, 2024. "Delivering Remote Learning Using a Low-Tech Solution: Evidence from a Randomized Controlled Trial in Bangladesh," Journal of Political Economy Microeconomics, University of Chicago Press, vol. 2(3), pages 562-601.
    3. Aparicio, Juan P. & Cook, Nikolai & Mikola, Derek & Rogeberg, Ole & Valenta, David & Wiebe, Michael & Bonander, Carl & Brodeur, Abel, 2025. "Comment on "Telementoring and Homeschooling during School Closures: A Randomized Experiment in Rural Bangladesh" by Hassan et al," I4R Discussion Paper Series 240, The Institute for Replication (I4R).
    4. Chemin, Matthieu & Schneider, Jeremy, 2025. "Online tutoring reduces by half the learning loss due to school closures: Evidence from a randomized experiment in Kenya," International Journal of Educational Development, Elsevier, vol. 117(C).
    5. Jakubowski,Maciej Jan & Gajderowicz,Tomasz Janusz & Patrinos,Harry Anthony, 2024. "COVID-19, School Closures, and Student Learning Outcomes : New Global Evidence from PISA," Policy Research Working Paper Series 10666, The World Bank.
    6. Lucas Gortazar & Claudia Hupkau & Antonio Roldan, 2023. "Online tutoring works: experimental evidence from a program with vulnerable children," CEP Discussion Papers dp1908, Centre for Economic Performance, LSE.
    7. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    8. Silke Anger & Bernhard Christoph & Agata Galkiewicz & Shushanik Margaryan & Malte Sandner & Thomas Siedler, 2025. "Online Tutoring, School Performance, and School-to-Work Transitions: Evidence from a Randomized Controlled Trial," Berlin School of Economics Discussion Papers 0084, Berlin School of Economics.
    9. Hashibul Hassan & Asad Islam & Abu Siddique & Liang Choon Wang, 2024. "Telementoring and Homeschooling During School Closures: a Randomised Experiment in Rural Bangladesh," The Economic Journal, Royal Economic Society, vol. 134(662), pages 2418-2438.
    10. Abhijeet Singh & Mauricio Romero & Karthik Muralidharan, 2022. "Covid-19 Learning Loss and Recovery: Panel Data Evidence from India," NBER Working Papers 30552, National Bureau of Economic Research, Inc.
    11. Gajderowicz, Tomasz & Jakubowski, Maciej & Kennedy, Alec & Christrup Kjeldsen, Christian & Patrinos, Harry Anthony & Strietholt, Rolf, 2024. "The Learning Crisis: Three Years After Covid-19," GLO Discussion Paper Series 1543, Global Labor Organization (GLO).
    12. Sakaue, Katsuki & Wokadala, James & Ogawa, Keiichi, 2023. "Effect of parental engagement on children’s home-based continued learning during COVID-19–induced school closures: Evidence from Uganda," International Journal of Educational Development, Elsevier, vol. 100(C).
    13. Aulagnon, Raphaëlle & Cristia, Julian & Cueto, Santiago & Malamud, Ofer, 2025. "Streaks to success: The effects of highlighting streaks on student effort and learning," Economics of Education Review, Elsevier, vol. 109(C).
    14. Cristia, Julian P. & Cueto, Santiago & Malamud, Ofer & Aulagnon, Raphaëlle, 2024. "Streaking to Success: The Effects of Highlighting Streaks on Student Effort and Achievement," IDB Publications (Working Papers) 13498, Inter-American Development Bank.
    15. Zoido, Pablo & Flores-Ceceña, Iván & Székely, Miguel & Hevia, Felipe J. & Castro, Eleno, 2024. "Remote tutoring with low-tech means to accelerate learning: Evidence for El Salvador," Economics of Education Review, Elsevier, vol. 98(C).
    16. Hardt, David & Nagler, Markus & Rincke, Johannes, 2023. "Tutoring in (online) higher education: Experimental evidence," Economics of Education Review, Elsevier, vol. 92(C).
    17. Alicia Brandt & Hamid R. Oskorouchi & Alfonso Sousa-Poza, 2024. "The effect of SMS nudges on higher education performance," Empirical Economics, Springer, vol. 66(5), pages 2311-2334, May.
    18. Harry Anthony Patrinos, 2022. "Learning loss and learning recovery," DECISION: Official Journal of the Indian Institute of Management Calcutta, Springer;Indian Institute of Management Calcutta, vol. 49(2), pages 183-188, June.
    19. Wang, Deliang & Zhen, Yuanyi & Shan, Dapeng & Zhang, Chenwei & Kao, Ben & Chen, Gaowei, 2025. "More investment, more engagement? Analyzing dialogic behavior in online one-to-one tutoring among different payment mechanisms," International Journal of Educational Development, Elsevier, vol. 117(C).
    20. Elisabetta Aurino & Sharon Wolf, 2024. "A ‘smart buy' for all? Unequal and unintended consequences of a messaging program for child education," UB School of Economics Working Papers 2024/461, University of Barcelona School of Economics.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I24 - Health, Education, and Welfare - - Education - - - Education and Inequality

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:i4rdps:241. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://www.i4replication.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.