IDEAS home Printed from https://ideas.repec.org/
MyIDEAS: Log in (now much improved!) to save this paper

Scaling-up What Works: Experimental Evidence on External Validity in Kenyan Education

Author Info

Listed author(s):
  • Tessa Bold
  • Mwangi Kimenyi
  • Germano Mwabu
  • Alice Ng'ang'a
  • Justin Sandefur

Abstract

The recent wave of randomized trials in development economics has provoked criticisms regarding external validity. We investigate two concerns – heterogeneity across beneficiaries and implementers – in a randomized trial of contract teachers in Kenyan schools. The intervention, previously shown to raise test scores in NGO-led trials in Western Kenya and parts of India, was replicated across all Kenyan provinces by an NGO and the government. Strong effects of short-term contracts produced in controlled experimental settings are lost in weak public institutions: NGO implementation produces a positive effect on test scores across diverse contexts, while government implementation yields zero effect. The data suggests that the stark contrast in success between the government and NGO arm can be traced back to implementation constraints and political economy forces put in motion as the program went to scale.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.csae.ox.ac.uk/workingpapers/pdfs/csae-wps-2013-04.pdf
Download Restriction: no

Paper provided by Centre for the Study of African Economies, University of Oxford in its series CSAE Working Paper Series with number 2013-04.

as
in new window

Length:
Date of creation: 2013
Handle: RePEc:csa:wpaper:2013-04
Contact details of provider: Postal:
Manor Road, Oxford, OX1 3UQ

Phone: +44-(0)1865 271084
Fax: +44-(0)1865 281447
Web page: http://www.csae.ox.ac.uk/
Email:


More information through EDIRC

Keywords:

This paper has been announced in the following NEP Reports:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

as
in new window


  1. Miriam Bruhn & David McKenzie, 2009. "In Pursuit of Balance: Randomization in Practice in Development Field Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
  2. Sarah Baird & Craig McIntosh & Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer Experiment," The Quarterly Journal of Economics, Oxford University Press, vol. 126(4), pages 1709-1753.
  3. Paxson, Christina & Schady, Norbert, 2007. "Does money matter ? The effects of cash transfers on child health and development in rural Ecuador," Policy Research Working Paper Series 4226, The World Bank.
  4. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
  5. Abhijit V. Banerjee & Shawn Cole & Esther Duflo & Leigh Linden, 2007. "Remedying Education: Evidence from Two Randomized Experiments in India," The Quarterly Journal of Economics, Oxford University Press, vol. 122(3), pages 1235-1264.
  6. Kazianga, Harounan & de Walque, Damien & Alderman, Harold, 2009. "Educational and health impacts of two school feeding schemes : evidence from a randomized trial in rural Burkina Faso," Policy Research Working Paper Series 4976, The World Bank.
  7. Esther Duflo & Pascaline Dupas & Michael Kremer, 2011. "Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya," American Economic Review, American Economic Association, vol. 101(5), pages 1739-1774, August.
  8. Barrera-Osorio, Felipe & Linden, Leigh L., 2009. "The use and misuse of computers in education : evidence from a randomized experiment in Colombia," Policy Research Working Paper Series 4836, The World Bank.
  9. Ritva Reinikka & Jakob Svensson, 2005. "Fighting Corruption to Improve Schooling: Evidence from a Newspaper Campaign in Uganda," Journal of the European Economic Association, MIT Press, vol. 3(2-3), pages 259-267, 04/05.
  10. Michael Kremer & Edward Miguel & Rebecca Thornton, 2009. "Incentives to Learn," The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 437-456, August.
  11. James Berry, 2015. "Child Control in Education Decisions: An Evaluation of Targeted Incentives to Learn in India," Journal of Human Resources, University of Wisconsin Press, vol. 50(4), pages 1051-1080.
  12. Tessa Bold, Mwangi Kimenyi, Germano Mwabu, Justin Sandefur, 2011. " Why Did Abolishing Fees Not Increase Public School Enrollment in Kenya?- Working Paper 271," Working Papers 271, Center for Global Development.
  13. Paul Atherton & Geeta Kingdon, 2010. "The relative effectiveness and costs of contract and regular teachers in India," CSAE Working Paper Series 2010-15, Centre for the Study of African Economies, University of Oxford.
  14. Dana Burde & Leigh L. Linden, 2012. "The Effect of Village-Based Schools: Evidence from a Randomized Controlled Trial in Afghanistan," NBER Working Papers 18039, National Bureau of Economic Research, Inc.
  15. Karthik Muralidharan & Venkatesh Sundararaman, 2013. "Contract Teachers: Experimental Evidence from India," NBER Working Papers 19440, National Bureau of Economic Research, Inc.
Full references (including those not matched with items on IDEAS)

Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

as
in new window


Cited by:

  1. Adrien Bouguen & Deon Filmer & Karen Macours & Sophie Naudeau, 2014. "Preschool and Parental Response in a Second Best World: Evidence from a School Construction Experiment," PSE Working Papers hal-01090361, HAL.
  2. Eva Vivalt, 2015. "Heterogeneous Treatment Effects in Impact Evaluation," American Economic Review, American Economic Association, vol. 105(5), pages 467-470, May.
  3. Michael Woolcock, 2013. "Using Case Studies to Explore the External Validity of ‘Complex’ Development Interventions," CID Working Papers 270, Center for International Development at Harvard University.
  4. Hunt Allcott, 2012. "Site Selection Bias in Program Evaluation," NBER Working Papers 18373, National Bureau of Economic Research, Inc.
  5. Karthik Muralidharan & Venkatesh Sundararaman, 2013. "Contract Teachers: Experimental Evidence from India," NBER Working Papers 19440, National Bureau of Economic Research, Inc.
  6. Paloyo, Alfredo R. & Rogan, Sally & Siminski, Peter, 2016. "The effect of supplemental instruction on academic performance: An encouragement design experiment," Economics of Education Review, Elsevier, vol. 55(C), pages 57-69.
  7. Kono, Hisaki & Sawada, Yasuyuki & Shonchoy, Abu S., 2016. "DVD-based distance-learning program for university entrance exams -- RCT experiments in rural Bangladesh," IDE Discussion Papers 580, Institute of Developing Economies, Japan External Trade Organization(JETRO).
  8. Sebastian Galiani & Patrick J. McEwan & Brian Quistorff, 2016. "External and Internal Validity of a Geographic Quasi-Experiment Embedded in Cluster-Randomized Experiment," NBER Working Papers 22468, National Bureau of Economic Research, Inc.
  9. Bouguen, Adrien & Filmer, Deon & Macours, Karen & Naudeau, Sophie, 2014. "Preschools and early childhood development in a second best world: Evidence from a scaled-up experiment in Cambodia," CEPR Discussion Papers 10170, C.E.P.R. Discussion Papers.
  10. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
  11. Delavallade, Clara & Griffith, Alan & Shukla, Gaurav & Thornton, Rebecca, 2017. "Participation, learning, and equity in education: Can we have it all?," IFPRI discussion papers 1627, International Food Policy Research Institute (IFPRI).
  12. Dhaliwal, Iqbal & Hanna, Rema, 2017. "The devil is in the details: The successes and limitations of bureaucratic reform in India," Journal of Development Economics, Elsevier, vol. 124(C), pages 1-21.
  13. Michael Clemens, Gabriel Demombynes, 2013. "The New Transparency in Development Economics: Lessons from the Millennium Villages Controversy," Working Papers 342, Center for Global Development.
  14. Olsson, Ola & Valsecchi, Michele, 2015. "Resource Windfalls and Local Government Behaviour: Evidence From a Policy Reform in Indonesia," Working Papers in Economics 635, University of Gothenburg, Department of Economics.
  15. Florent Bedecarrats & Isabelle Guérin & François Roubaud, 2017. "L'étalon-or des évaluations randomisées : du discours de la méthode à l'économie politique," Working Papers ird-01445209, HAL.
  16. Lant Pritchett, Justin Sandefur, 2013. "Context Matters for Size: Why External Validity Claims and Development Practice Don't Mix-Working Paper 336," Working Papers 336, Center for Global Development.
  17. Lant Pritchett & Justin Sandefur, 2015. "Learning from Experiments When Context Matters," American Economic Review, American Economic Association, vol. 105(5), pages 471-475, May.
  18. Barrera-Osorio, Felipe & Raju, Dhushyanth, 2017. "Teacher performance pay: Experimental evidence from Pakistan," Journal of Public Economics, Elsevier, vol. 148(C), pages 75-91.

This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

Access and download statistics

When requesting a correction, please mention this item's handle: RePEc:csa:wpaper:2013-04. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Payne)

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.

This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.