IDEAS home Printed from https://ideas.repec.org/p/ags/umaesp/316663.html
   My bibliography  Save this paper

Why Programs Fail: Lessons for Improving Public Service Quality from a Mixed-Methods Evaluation of an Unsuccessful Teacher Training Program in Nepal

Author

Listed:
  • Schaffner, Julie
  • Glewwe, Paul
  • Sharma, Uttam

Abstract

Using a randomized control trial embedded within a mixed-methods evaluation, we find that an at-scale government teacher training program, of a common but seldom-evaluated form, has little or no impact on student learning. We then document five challenges that the policy’s design failed to address, related to: oversight of training sessions, school-level difficulties in releasing teachers for training (lack of substitute teachers), deficits in teachers’ subject knowledge, deficits in teachers’ post-training accountability and support, and students’ needs for differentiated instruction. We discuss implications for the literatures on teacher training program design and on good governance for public service provision.

Suggested Citation

  • Schaffner, Julie & Glewwe, Paul & Sharma, Uttam, 2021. "Why Programs Fail: Lessons for Improving Public Service Quality from a Mixed-Methods Evaluation of an Unsuccessful Teacher Training Program in Nepal," Staff Papers 316663, University of Minnesota, Department of Applied Economics.
  • Handle: RePEc:ags:umaesp:316663
    DOI: 10.22004/ag.econ.316663
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/316663/files/WhyProgramsFailP21_3.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.22004/ag.econ.316663?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    2. Annie Duflo & Jessica Kiessel & Adrienne Lucas, 2020. "Experimental Evidence on Alternative Policies to Increase Learning at Scale," NBER Working Papers 27298, National Bureau of Economic Research, Inc.
    3. Gauri, Varun & Jamison, Julian C. & Mazar, Nina & Ozier, Owen, 2021. "Motivating bureaucrats through social recognition: External validity—A tale of two states," Organizational Behavior and Human Decision Processes, Elsevier, vol. 163(C), pages 117-131.
    4. Anna Popova & David K Evans & Mary E Breeding & Violeta Arancibia, 2022. "Teacher Professional Development around the World: The Gap between Evidence and Practice [Training to Teach Science: Experimental Evidence from Argentina]," The World Bank Research Observer, World Bank, vol. 37(1), pages 107-136.
    5. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    6. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2010. "Teacher Credentials and Student Achievement in High School: A Cross-Subject Analysis with Student Fixed Effects," Journal of Human Resources, University of Wisconsin Press, vol. 45(3).
    7. Linxiu Zhang & Fang Lai & Xiaopeng Pang & Hongmei Yi & Scott Rozelle, 2013. "The impact of teacher training on teacher and student outcomes: evidence from a randomised experiment in Beijing migrant schools," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 5(3), pages 339-358, September.
    8. Jacobus Cilliers & Brahm Fleisch & Cas Prinsloo & Stephen Taylor, 2020. "How to Improve Teaching Practice?: An Experimental Comparison of Centralized Training and In-Classroom Coaching," Journal of Human Resources, University of Wisconsin Press, vol. 55(3), pages 926-962.
    9. Scur, Daniela & Lemos, Renata & Muralidharan, Karthik, 2021. "Personnel Management and School Productivity: Evidence from India," CEPR Discussion Papers 15659, C.E.P.R. Discussion Papers.
    10. Glewwe, P. & Muralidharan, K., 2016. "Improving Education Outcomes in Developing Countries," Handbook of the Economics of Education,, Elsevier.
    11. Habtamu Fuje & Prateek Tandon, 2018. "When do in‐service teacher training and books improve student achievement? Experimental evidence from Mongolia," Review of Development Economics, Wiley Blackwell, vol. 22(3), pages 1360-1383, August.
    12. Blimpo, Moussa P. & Pugatch, Todd, 2021. "Entrepreneurship education and teacher training in Rwanda," Journal of Development Economics, Elsevier, vol. 149(C).
    13. Asian Development Bank Institute, 2017. "Innovative Strategies in Higher Education for Accelerated Human Resource Development in South Asia: Nepal," Working Papers id:11816, eSocialSciences.
    14. Duflo, Esther & Banerjee, Abhijit & Banerji, Rukmini & Berry, James & Mukerji, Shobhini & Shotland, Marc & Walton, Michael & Kannan, Harini, 2016. "Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of “Teaching at the Right Level†in India," CEPR Discussion Papers 11530, C.E.P.R. Discussion Papers.
    15. Banerjee, Abjijit & Banerji, Rukmini & Berry, James & Duflo, Esther & Kannan, Harini & Mukerji, Shobhini & Shotland, Marc & Walton, Michael, 2016. "Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of "Teaching at the Right Level" in India," Working Paper Series rwp16-043, Harvard University, John F. Kennedy School of Government.
    16. Adrienne M. Lucas & Patrick J. McEwan & Moses Ngware & Moses Oketch, 2014. "Improving Early‐Grade Literacy In East Africa: Experimental Evidence From Kenya And Uganda," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(4), pages 950-976, September.
    17. World Bank, 2018. "World Development Report 2018 [Rapport sur le développement dans le monde 2018]," World Bank Publications - Books, The World Bank Group, number 28340, December.
    18. Prashant Loyalka & Anna Popova & Guirong Li & Zhaolei Shi, 2019. "Does Teacher Training Actually Work? Evidence from a Large-Scale Randomized Evaluation of a National Teacher Training Program," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 128-154, July.
    19. Macdonald,Kevin Alan David & Brinkman,Sally Anne & Jarvie,Wendy & Machuca-Sierra,Myrna & Mcdonall,Kristen Andrew & Messaoud-Galusi,Souhila & Tapueluelu,Siosiana & Vu,Binh Thanh, 2018. "Intervening at Home and Then at School : A Randomized Evaluation of Two Approaches to Improve Early Educational Outcomes in Tonga," Policy Research Working Paper Series 8682, The World Bank.
    20. Rosangela Bando & Emma Näslund-Hadley & Paul Gertler, 2019. "Effect of Inquiry and Problem Based Pedagogy on Learning: Evidence from 10 Field Experiments in Four Countries," NBER Working Papers 26280, National Bureau of Economic Research, Inc.
    21. Piper, Benjamin & Simmons Zuilkowski, Stephanie & Dubeck, Margaret & Jepkemei, Evelyn & King, Simon J., 2018. "Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides," World Development, Elsevier, vol. 106(C), pages 324-336.
    22. Bruns, Barbara & Costa, Leandro & Cunha, Nina, 2018. "Through the looking glass: Can classroom observation and coaching improve teacher performance in Brazil?," Economics of Education Review, Elsevier, vol. 64(C), pages 214-250.
    23. Cilliers, Jacobus & Fleisch, Brahm & Kotze, Janeli & Mohohlwane, Nompumelelo & Taylor, Stephen & Thulare, Tsegofatso, 2022. "Can virtual replace in-person coaching? Experimental evidence on teacher professional development and student learning," Journal of Development Economics, Elsevier, vol. 155(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Juan F. Castro & Paul Glewwe & Ricardo Montero, 2019. "Work With What You’ve Got: Improving Teachers’ Pedagogical Skills at Scale in Rural Peru," Working Papers 158, Peruvian Economic Association.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Blimpo, Moussa P. & Pugatch, Todd, 2021. "Entrepreneurship education and teacher training in Rwanda," Journal of Development Economics, Elsevier, vol. 149(C).
    2. Cruz-Aguayo, Yyannú & Carneiro, Pedro & Intriago, Ruthy & Ponce, Juan & Schady, Norbert & Schodt, Sarah, 2022. "When Promising Interventions Fail: Personalized Coaching for Teachers in a Middle-Income Country," IZA Discussion Papers 15021, Institute of Labor Economics (IZA).
    3. Johnston, Jamie & Ksoll, Christopher, 2022. "Effectiveness of interactive satellite-transmitted instruction: Experimental evidence from Ghanaian primary schools," Economics of Education Review, Elsevier, vol. 91(C).
    4. Blimpo, Moussa P. & Pugatch, Todd, 2023. "Unintended Consequences of Youth Entrepreneurship Programs: Experimental Evidence from Rwanda," GLO Discussion Paper Series 1332, Global Labor Organization (GLO).
    5. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    6. Suzanne Bellue & Adrien Bouguen & Marc Gurand & Valerie Munier & André Tricot, 2023. "When Effective Teacher Training Falls Short in the Classroom: Evidence from an Experiment in Primary Schools," CRC TR 224 Discussion Paper Series crctr224_2023_455, University of Bonn and University of Mannheim, Germany.
    7. Angrist, Noam & de Barros, Andreas & Bhula, Radhika & Chakera, Shiraz & Cummiskey, Chris & DeStefano, Joseph & Floretta, John & Kaffenberger, Michelle & Piper, Benjamin & Stern, Jonathan, 2021. "Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery," International Journal of Educational Development, Elsevier, vol. 84(C).
    8. Juan F. Castro & Paul Glewwe & Ricardo Montero, 2019. "Work With What You’ve Got: Improving Teachers’ Pedagogical Skills at Scale in Rural Peru," Working Papers 158, Peruvian Economic Association.
    9. Buhl-Wiggers, Julie & Kerwin, Jason & Muñoz-Morales, Juan S. & Smith, Jeffrey A. & Thornton, Rebecca L., 2020. "Some Children Left Behind: Variation in the Effects of an Educational Intervention," IZA Discussion Papers 13598, Institute of Labor Economics (IZA).
    10. Aymo Brunetti & Konstantin Büchel & Martina Jakob & Ben Jann & Daniel Steffen, 2021. "Inadequate Teacher Content Knowledge and What to Do About It: Evidence from El Salvador," University of Bern Social Sciences Working Papers 41, University of Bern, Department of Social Sciences.
    11. Annie Duflo & Jessica Kiessel & Adrienne Lucas, 2020. "Experimental Evidence on Alternative Policies to Increase Learning at Scale," NBER Working Papers 27298, National Bureau of Economic Research, Inc.
    12. Hanushek, Eric A., 2021. "Addressing cross-national generalizability in educational impact evaluation," International Journal of Educational Development, Elsevier, vol. 80(C).
    13. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    14. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    15. Jacobus Cilliers & Brahm Fleisch & Cas Prinsloo & Stephen Taylor, 2020. "How to Improve Teaching Practice?: An Experimental Comparison of Centralized Training and In-Classroom Coaching," Journal of Human Resources, University of Wisconsin Press, vol. 55(3), pages 926-962.
    16. Yasuyuki Sawada & Minhaj Mahmud & Mai Seki & An Le & Hikaru Kawarazaki, 2019. "Fighting the Learning Crisis in Developing Countries: A Randomized Experiment of Self-Learning at the Right Level," CIRJE F-Series CIRJE-F-1127, CIRJE, Faculty of Economics, University of Tokyo.
    17. Francesca Marchetta & Tom Dilly, 2019. "Supporting Education in Africa: Opportunities and Challenges for an Impact Investor," Working Papers hal-02288103, HAL.
    18. Anand, Gautam & Atluri, Aishwarya & Crawfurd, Lee & Pugatch, Todd & Sheth, Ketki, 2023. "Improving school management in low and middle income countries: A systematic review," Economics of Education Review, Elsevier, vol. 97(C).
    19. Derksen, Laura & Michaud-Leclerc, Catherine & Souza, Pedro C.L., 2022. "Restricted access: How the internet can be used to promote reading and learning," Journal of Development Economics, Elsevier, vol. 155(C).
    20. Jason T. Kerwin & Rebecca L. Thornton, 2021. "Making the Grade: The Sensitivity of Education Program Effectiveness to Input Choices and Outcome Measures," The Review of Economics and Statistics, MIT Press, vol. 103(2), pages 251-264, May.

    More about this item

    Keywords

    Community/Rural/Urban Development; Public Economics; Teaching/Communication/Extension/Profession;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:umaesp:316663. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/daumnus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.