IDEAS home Printed from https://ideas.repec.org/p/ags/umaesp/316663.html

Why Programs Fail: Lessons for Improving Public Service Quality from a Mixed-Methods Evaluation of an Unsuccessful Teacher Training Program in Nepal

Author

Listed:
  • Schaffner, Julie
  • Glewwe, Paul
  • Sharma, Uttam

Abstract

Using a randomized control trial embedded within a mixed-methods evaluation, we find that an at-scale government teacher training program, of a common but seldom-evaluated form, has little or no impact on student learning. We then document five challenges that the policy’s design failed to address, related to: oversight of training sessions, school-level difficulties in releasing teachers for training (lack of substitute teachers), deficits in teachers’ subject knowledge, deficits in teachers’ post-training accountability and support, and students’ needs for differentiated instruction. We discuss implications for the literatures on teacher training program design and on good governance for public service provision.

Suggested Citation

  • Schaffner, Julie & Glewwe, Paul & Sharma, Uttam, 2021. "Why Programs Fail: Lessons for Improving Public Service Quality from a Mixed-Methods Evaluation of an Unsuccessful Teacher Training Program in Nepal," Staff Papers 316663, University of Minnesota, Department of Applied Economics.
  • Handle: RePEc:ags:umaesp:316663
    DOI: 10.22004/ag.econ.316663
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/316663/files/WhyProgramsFailP21_3.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.22004/ag.econ.316663?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Annie Duflo & Jessica Kiessel & Adrienne Lucas, 2020. "Experimental Evidence on Alternative Policies to Increase Learning at Scale," NBER Working Papers 27298, National Bureau of Economic Research, Inc.
    2. Sabrin Beg & Waqas Halim & Adrienne M. Lucas & Umar Saif, 2022. "Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 61-90, May.
    3. Gauri, Varun & Jamison, Julian C. & Mazar, Nina & Ozier, Owen, 2021. "Motivating bureaucrats through social recognition: External validity—A tale of two states," Organizational Behavior and Human Decision Processes, Elsevier, vol. 163(C), pages 117-131.
    4. Anna Popova & David K Evans & Mary E Breeding & Violeta Arancibia, 2022. "Teacher Professional Development around the World: The Gap between Evidence and Practice [Training to Teach Science: Experimental Evidence from Argentina]," The World Bank Research Observer, World Bank, vol. 37(1), pages 107-136.
    5. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    6. Linxiu Zhang & Fang Lai & Xiaopeng Pang & Hongmei Yi & Scott Rozelle, 2013. "The impact of teacher training on teacher and student outcomes: evidence from a randomised experiment in Beijing migrant schools," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 5(3), pages 339-358, September.
    7. Jacobus Cilliers & Brahm Fleisch & Cas Prinsloo & Stephen Taylor, 2020. "How to Improve Teaching Practice?: An Experimental Comparison of Centralized Training and In-Classroom Coaching," Journal of Human Resources, University of Wisconsin Press, vol. 55(3), pages 926-962.
    8. Renata Lemos & Karthik Muralidharan & Daniela Scur, 2024. "Personnel Management and School Productivity: Evidence from India," The Economic Journal, Royal Economic Society, vol. 134(661), pages 2071-2100.
    9. Glewwe, P. & Muralidharan, K., 2016. "Improving Education Outcomes in Developing Countries," Handbook of the Economics of Education,, Elsevier.
    10. Habtamu Fuje & Prateek Tandon, 2018. "When do in‐service teacher training and books improve student achievement? Experimental evidence from Mongolia," Review of Development Economics, Wiley Blackwell, vol. 22(3), pages 1360-1383, August.
    11. Duflo, Esther & Banerjee, Abhijit & Banerji, Rukmini & Berry, James & Mukerji, Shobhini & Shotland, Marc & Walton, Michael & Kannan, Harini, 2016. "Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of “Teaching at the Right Level†in India," CEPR Discussion Papers 11530, Centre for Economic Policy Research.
    12. Asian Development Bank Institute, 2017. "Innovative Strategies in Higher Education for Accelerated Human Resource Development in South Asia: Nepal," Working Papers id:11816, eSocialSciences.
    13. Banerjee, Abjijit & Banerji, Rukmini & Berry, James & Duflo, Esther & Kannan, Harini & Mukerji, Shobhini & Shotland, Marc & Walton, Michael, 2016. "Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of "Teaching at the Right Level" in India," Working Paper Series rwp16-043, Harvard University, John F. Kennedy School of Government.
    14. Adrienne M. Lucas & Patrick J. McEwan & Moses Ngware & Moses Oketch, 2014. "Improving Early‐Grade Literacy In East Africa: Experimental Evidence From Kenya And Uganda," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 33(4), pages 950-976, September.
    15. repec:wbk:wbpubs:28340 is not listed on IDEAS
    16. Prashant Loyalka & Anna Popova & Guirong Li & Zhaolei Shi, 2019. "Does Teacher Training Actually Work? Evidence from a Large-Scale Randomized Evaluation of a National Teacher Training Program," American Economic Journal: Applied Economics, American Economic Association, vol. 11(3), pages 128-154, July.
    17. Macdonald,Kevin Alan David & Brinkman,Sally Anne & Jarvie,Wendy & Machuca-Sierra,Myrna & Mcdonall,Kristen Andrew & Messaoud-Galusi,Souhila & Tapueluelu,Siosiana & Vu,Binh Thanh, 2018. "Intervening at Home and Then at School : A Randomized Evaluation of Two Approaches to Improve Early Educational Outcomes in Tonga," Policy Research Working Paper Series 8682, The World Bank.
    18. Rosangela Bando & Emma Näslund-Hadley & Paul Gertler, 2019. "Effect of Inquiry and Problem Based Pedagogy on Learning: Evidence from 10 Field Experiments in Four Countries," NBER Working Papers 26280, National Bureau of Economic Research, Inc.
    19. Piper, Benjamin & Simmons Zuilkowski, Stephanie & Dubeck, Margaret & Jepkemei, Evelyn & King, Simon J., 2018. "Identifying the essential ingredients to literacy and numeracy improvement: Teacher professional development and coaching, student textbooks, and structured teachers’ guides," World Development, Elsevier, vol. 106(C), pages 324-336.
    20. Berry, James & Kannan, Harini & Mukherji, Shobhini & Shotland, Marc, 2020. "Failure of frequent assessment: An evaluation of India’s continuous and comprehensive evaluation program," Journal of Development Economics, Elsevier, vol. 143(C).
    21. Charles T. Clotfelter & Helen F. Ladd & Jacob L. Vigdor, 2010. "Teacher Credentials and Student Achievement in High School: A Cross-Subject Analysis with Student Fixed Effects," Journal of Human Resources, University of Wisconsin Press, vol. 45(3).
    22. Blimpo, Moussa P. & Pugatch, Todd, 2021. "Entrepreneurship education and teacher training in Rwanda," Journal of Development Economics, Elsevier, vol. 149(C).
    23. Bruns, Barbara & Costa, Leandro & Cunha, Nina, 2018. "Through the looking glass: Can classroom observation and coaching improve teacher performance in Brazil?," Economics of Education Review, Elsevier, vol. 64(C), pages 214-250.
    24. Cilliers, Jacobus & Fleisch, Brahm & Kotze, Janeli & Mohohlwane, Nompumelelo & Taylor, Stephen & Thulare, Tsegofatso, 2022. "Can virtual replace in-person coaching? Experimental evidence on teacher professional development and student learning," Journal of Development Economics, Elsevier, vol. 155(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Juan F. Castro & Paul Glewwe & Ricardo Montero, 2019. "Work With What You’ve Got: Improving Teachers’ Pedagogical Skills at Scale in Rural Peru," Working Papers 158, Peruvian Economic Association.
    2. Biswas, Kumar & de Galbert, Pierre & Sabarwal, Shwetlena & Glave, Carla Z. & Asaduzzaman, T.M., 2025. "Online training and financial incentives for teachers: Evidence from Bangladesh," Journal of Economic Behavior & Organization, Elsevier, vol. 229(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Blimpo, Moussa P. & Pugatch, Todd, 2021. "Entrepreneurship education and teacher training in Rwanda," Journal of Development Economics, Elsevier, vol. 149(C).
    2. Johnston, Jamie & Ksoll, Christopher, 2022. "Effectiveness of interactive satellite-transmitted instruction: Experimental evidence from Ghanaian primary schools," Economics of Education Review, Elsevier, vol. 91(C).
    3. Amaro Da Costa Luz Carneiro,Pedro Manuel & Cruz-Aguayo,Yyannu & Intriago,Ruthy & Ponce,Juan & Schady,Norbert Rudiger & Schodt,Sarah, 2022. "When Promising Interventions Fail : Personalized Coaching for Teachers in a Middle-Income Country," Policy Research Working Paper Series 9926, The World Bank.
    4. Bellue, Suzanne & Bouguen, Adrien & Gurgand, Marc & Munier, Valerie & Tricot, André, 2024. "When Effective teacher training falls short in the classroom: Evidence from an experiment in primary schools," Economics of Education Review, Elsevier, vol. 103(C).
    5. Juan F. Castro & Paul Glewwe & Alexandra Heredia‐Mayo & Stephanie Majerowicz & Ricardo Montero, 2025. "Can teaching be taught? Improving teachers' pedagogical skills at scale in rural Peru," Quantitative Economics, Econometric Society, vol. 16(1), pages 185-233, January.
    6. Blimpo, Moussa P. & Pugatch, Todd, 2023. "Unintended Consequences of Youth Entrepreneurship Programs: Experimental Evidence from Rwanda," GLO Discussion Paper Series 1332, Global Labor Organization (GLO).
    7. Juan F. Castro & Paul Glewwe & Ricardo Montero, 2019. "Work With What You’ve Got: Improving Teachers’ Pedagogical Skills at Scale in Rural Peru," Working Papers 158, Peruvian Economic Association.
    8. Annie Duflo & Jessica Kiessel & Adrienne Lucas, 2020. "Experimental Evidence on Alternative Policies to Increase Learning at Scale," NBER Working Papers 27298, National Bureau of Economic Research, Inc.
    9. Bharti, Nitin & Malik, Samreen & Mukhopadhyay, Abhiroop & Prakash, Nishith, 2026. "Science on the Move: How Experiential Pedagogy Shapes Human Capital," IZA Discussion Papers 18677, IZA Network @ LISER.
    10. Karthik Muralidharan & Abhijeet Singh & Alejandro J. Ganimian, 2019. "Disrupting Education? Experimental Evidence on Technology-Aided Instruction in India," American Economic Review, American Economic Association, vol. 109(4), pages 1426-1460, April.
    11. Angrist, Noam & de Barros, Andreas & Bhula, Radhika & Chakera, Shiraz & Cummiskey, Chris & DeStefano, Joseph & Floretta, John & Kaffenberger, Michelle & Piper, Benjamin & Stern, Jonathan, 2021. "Building back better to avert a learning catastrophe: Estimating learning loss from COVID-19 school shutdowns in Africa and facilitating short-term and long-term learning recovery," International Journal of Educational Development, Elsevier, vol. 84(C).
    12. Jason T. Kerwin & Rebecca L. Thornton, 2021. "Making the Grade: The Sensitivity of Education Program Effectiveness to Input Choices and Outcome Measures," The Review of Economics and Statistics, MIT Press, vol. 103(2), pages 251-264, May.
    13. Bergman, Peter & Chowanajin, Nat, 2026. "Characterizing the File Drawer: Evidence from a Meta-Analysis of Parent-Interventions Around the World," IZA Discussion Papers 18701, IZA Network @ LISER.
    14. Berlinski, Samuel & Giannola, Michele & Toppeta, Alessandro, 2026. "Effective Families or Effective Schools? Experimental Evidence on Fostering Children’s Numeracy," SOFI Working Papers in Labour Economics 6/2026, Stockholm University, Swedish Institute for Social Research.
    15. Emily Beam & Priya Mukherjee & Laia Navarro-Sola, 2022. "Lowering Barriers to Remote Education: Experimental Impacts on Parental Responses and Learning," Working Papers 2022-030, Human Capital and Economic Opportunity Working Group.
    16. Anand, Gautam & Atluri, Aishwarya & Crawfurd, Lee & Pugatch, Todd & Sheth, Ketki, 2023. "Improving school management in low and middle income countries: A systematic review," Economics of Education Review, Elsevier, vol. 97(C).
    17. Susanti, Dewi & Anand, Gautam & Arifin, Firda Arianti, 2025. "Leveraging school principals to address learning loss in Indonesia through group and individual targeting," International Journal of Educational Development, Elsevier, vol. 112(C).
    18. Yue-Yi Hwa & Clare Leaver, 2021. "Management in education systems," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 37(2), pages 367-391.
    19. Aymo Brunetti & Konstantin Büchel & Martina Jakob & Ben Jann & Daniel Steffen, 2021. "Inadequate Teacher Content Knowledge and What to Do About It: Evidence from El Salvador," University of Bern Social Sciences Working Papers 41, University of Bern, Department of Social Sciences.
    20. Krafft,Caroline Gould & Nikaein Towfighian,Samira & Raikes,Abbie & Mojgani,Rebecca Sayre, 2023. "What Can We Learn from Pre-Primary Quality Assurance Systems ? Evidence from theArab Republic of Egypt," Policy Research Working Paper Series 10482, The World Bank.

    More about this item

    Keywords

    ;
    ;
    ;

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:umaesp:316663. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/daumnus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.