IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v66y2018icp73-91.html
   My bibliography  Save this article

Concentrating efforts on low-performing schools: Impact estimates from a quasi-experimental design

Author

Listed:
  • Barrera-Osorio, Felipe
  • García, Sandra
  • Rodríguez, Catherine
  • Sánchez, Fabio
  • Arbeláez, Mateo

Abstract

This paper presents the impact evaluation results of the Colombian program Todos a Aprender (Everyone Learning Program, ELP), a multi-level intervention targeting low-performing schools. The main objective of the program was to increase math and language test scores of these schools through on-site teacher training, principal training and textbooks for students. Using census data from public schools containing detailed longitudinal information since 2010, the starting year of the program, and taking advantage of targeting rules based on dropout and grade repetition rates we fit a fuzzy regression discontinuity design to estimate program impacts. We also fit a difference-in-difference matching model as well as blocking with regressions to estimate the ATT impact of the program, based on observed characteristics used in the targeting process. Overall results indicate no significant impact of the program on test scores, grade repetition nor dropout rates. Additional analyses from a representative sample of 400 schools collected in the field suggest that deficiencies in the program's design and implementation could explain the lack of significant program impacts.

Suggested Citation

  • Barrera-Osorio, Felipe & García, Sandra & Rodríguez, Catherine & Sánchez, Fabio & Arbeláez, Mateo, 2018. "Concentrating efforts on low-performing schools: Impact estimates from a quasi-experimental design," Economics of Education Review, Elsevier, vol. 66(C), pages 73-91.
    • Handle: RePEc:eee:ecoedu:v:66:y:2018:i:c:p:73-91
    DOI: 10.1016/j.econedurev.2018.07.001
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S027277571730568X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2018.07.001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Sebastian Calonico & Matias D. Cattaneo & Rocio Titiunik, 2014. "Robust Nonparametric Confidence Intervals for Regression‐Discontinuity Designs," Econometrica, Econometric Society, vol. 82, pages 2295-2326, November.
    3. Felipe Barrera-Osorio & Darío Maldonado & Catherine Rodríguez, 2012. "Calidad de la Educación Básica y Media en Colombia: Diagnóstico y Propuestas," Documentos CEDE 10321, Universidad de los Andes, Facultad de Economía, CEDE.
    4. Kenneth Y. Chay & Patrick J. McEwan & Miguel Urquiola, 2005. "The Central Role of Noise in Evaluating Interventions That Use Test Scores to Rank Schools," American Economic Review, American Economic Association, vol. 95(4), pages 1237-1258, September.
    5. Glewwe, Paul (ed.), 2013. "Education Policy in Developing Countries," University of Chicago Press Economics Books, University of Chicago Press, number 9780226078687, September.
    6. Battistin, Erich & Rettore, Enrico, 2008. "Ineligibles and eligible non-participants as a double comparison group in regression-discontinuity designs," Journal of Econometrics, Elsevier, vol. 142(2), pages 715-730, February.
    7. Paqueo, Vicente & Lopez-Acevedo, Gladys, 2003. "Supply-side school improvement and the learning achievement of the poorest children in indigenous and rural schools - the case of PARE," Policy Research Working Paper Series 3172, The World Bank.
    8. Guido W. Imbens, 2015. "Matching Methods in Practice: Three Examples," Journal of Human Resources, University of Wisconsin Press, vol. 50(2), pages 373-419.
    9. Papay, John P. & Willett, John B. & Murnane, Richard J., 2011. "Extending the regression-discontinuity approach to multiple assignment variables," Journal of Econometrics, Elsevier, vol. 161(2), pages 203-207, April.
    10. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    11. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    12. Alejandro J. Ganimian & Richard J. Murnane, 2014. "Improving Educational Outcomes in Developing Countries: Lessons from Rigorous Impact Evaluations," NBER Working Papers 20284, National Bureau of Economic Research, Inc.
    13. McCrary, Justin, 2008. "Manipulation of the running variable in the regression discontinuity design: A density test," Journal of Econometrics, Elsevier, vol. 142(2), pages 698-714, February.
    14. Rodríguez, Catherine & Sánchez, Fabio & Armenta, Armando, 2010. "Do Interventions at School Level Improve Educational Outcomes? Evidence from a Rural Program in Colombia," World Development, Elsevier, vol. 38(3), pages 415-428, March.
    15. Tavares, Priscilla Albuquerque, 2015. "The impact of school management practices on educational performance: Evidence from public schools in São Paulo," Economics of Education Review, Elsevier, vol. 48(C), pages 1-15.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marín Llanes, Lucas & Rodríguez Pico, Mariana & Maldonado, Darío & García, Sandra, 2023. "Learning inequality during Covid-19: Evidence from secondary schools in Colombia," International Journal of Educational Development, Elsevier, vol. 100(C).
    2. Sofía Collante Zárate & Catherine Rodríguez & Fabio Sanchez, 2022. "El poder de un refrigerio. La alimentación escolar y sus efectos educativos en Colombia," Documentos CEDE 20223, Universidad de los Andes, Facultad de Economía, CEDE.
    3. Parra, Juan David, 2022. "Decentralisation and school-based management in Colombia: An exploration (using systems thinking) of the Full‐Day Schooling programme," International Journal of Educational Development, Elsevier, vol. 91(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Deng, Taotao & Hu, Yukun & Ma, Mulan, 2019. "Regional policy and tourism: A quasi-natural experiment," Annals of Tourism Research, Elsevier, vol. 74(C), pages 1-16.
    2. Mellace, Giovanni & Ventura, Marco, 2019. "Intended and unintended effects of public incentives for innovation. Quasi-experimental evidence from Italy," Discussion Papers on Economics 9/2019, University of Southern Denmark, Department of Economics.
    3. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    4. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    5. Chad D. Meyerhoefer & Muzhe Yang, 2011. "The Relationship between Food Assistance and Health: A Review of the Literature and Empirical Strategies for Identifying Program Effects," Applied Economic Perspectives and Policy, Agricultural and Applied Economics Association, vol. 33(3), pages 304-344.
    6. Vanessa Cirulli & Giuliano Resce & Marco Ventura, 2021. "Co-payment exemption and healthcare consumption. Quasi-experimental evidence from Italy," Working Papers in Public Economics 203, University of Rome La Sapienza, Department of Economics and Law.
    7. Koch, Nicolas & Basse Mama, Houdou, 2019. "Does the EU Emissions Trading System induce investment leakage? Evidence from German multinational firms," Energy Economics, Elsevier, vol. 81(C), pages 479-492.
    8. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    9. Kämpfen, F.; & Gómez-Olivé, X.; & O’Donnell, O.; & Riumallo Herl, C.;, 2023. "Effectiveness of Population-Based Hypertension Screening: A Multidimensional Regression Discontinuity Design," Health, Econometrics and Data Group (HEDG) Working Papers 23/15, HEDG, c/o Department of Economics, University of York.
    10. Giuseppe Francesco Gori & Patrizia Lattarulo & Marco Mariani, 2021. "The Expediting Effect of Monitoring on Infrastructural Works. A Regression-Discontinuity Approach with Multiple Assignment Variables," Papers 2102.09625, arXiv.org.
    11. Guido Imbens & Stefan Wager, 2019. "Optimized Regression Discontinuity Designs," The Review of Economics and Statistics, MIT Press, vol. 101(2), pages 264-278, May.
    12. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    13. Wagener, Andreas & Zenker, Juliane, 2018. "Decoupled but not neutral: The effects of stochastic transfers on investment and incomes in rural Thailand," TVSEP Working Papers wp-008, Leibniz Universitaet Hannover, Institute of Development and Agricultural Economics, Project TVSEP.
    14. Marco Caliendo & Stefan Tübbicke, 2020. "New evidence on long-term effects of start-up subsidies: matching estimates and their robustness," Empirical Economics, Springer, vol. 59(4), pages 1605-1631, October.
    15. David S. Lee & Thomas Lemieux, 2009. "Regression Discontinuity Designs In Economics," Working Papers 1118, Princeton University, Department of Economics, Industrial Relations Section..
    16. Benjamin Lu & Eli Ben-Michael & Avi Feller & Luke Miratrix, 2023. "Is It Who You Are or Where You Are? Accounting for Compositional Differences in Cross-Site Treatment Effect Variation," Journal of Educational and Behavioral Statistics, , vol. 48(4), pages 420-453, August.
    17. Luís Felipe Batista de Oliveira & Rafael Terra, 2016. "Impact of school day extension on educational outcomes: evidence from Mais Educação in Brazil," One Pager 329, International Policy Centre for Inclusive Growth.
    18. Dana Rotz & Brian Goesling & Molly Crofton & Jennifer Manlove & Kate Welti, "undated". "Final Impacts of Teen PEP (Teen Prevention Education Program) in New Jersey and North Carolina High Schools," Mathematica Policy Research Reports 40fc2bb74d874e59a8e424638, Mathematica Policy Research.
    19. Flores, Carlos A. & Mitnik, Oscar A., 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," IZA Discussion Papers 4451, Institute of Labor Economics (IZA).
    20. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.

    More about this item

    Keywords

    Multi-level intervention; Regression discontinuity; Program design and implementation;
    All these keywords.

    JEL classification:

    • C29 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Other
    • C26 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Instrumental Variables (IV) Estimation
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • I28 - Health, Education, and Welfare - - Education - - - Government Policy
    • I29 - Health, Education, and Welfare - - Education - - - Other

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:66:y:2018:i:c:p:73-91. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.