IDEAS home Printed from https://ideas.repec.org/a/eee/ecoedu/v58y2017icp123-140.html
   My bibliography  Save this article

The impact of an accountability intervention with diagnostic feedback: Evidence from Mexico

Author

Listed:
  • de Hoyos, Rafael
  • Garcia-Moreno, Vicente A.
  • Patrinos, Harry Anthony

Abstract

The Mexican state of Colima implemented a low-stakes accountability intervention with diagnostic feedback among schools with the lowest test scores in the national assessment. A difference-in-difference and a regression discontinuity design are used to identify the effects of the intervention on learning outcomes. The two strategies consistently show that the intervention increased test scores by 0.12 standard deviations only a few months after the program was launched. The results indicate that full and wide dissemination of information detailing school quality is critically important.

Suggested Citation

  • de Hoyos, Rafael & Garcia-Moreno, Vicente A. & Patrinos, Harry Anthony, 2017. "The impact of an accountability intervention with diagnostic feedback: Evidence from Mexico," Economics of Education Review, Elsevier, vol. 58(C), pages 123-140.
    • Handle: RePEc:eee:ecoedu:v:58:y:2017:i:c:p:123-140
    DOI: 10.1016/j.econedurev.2017.03.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0272775715302478
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.econedurev.2017.03.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    2. Kenneth Y. Chay & Patrick J. McEwan & Miguel Urquiola, 2005. "The Central Role of Noise in Evaluating Interventions That Use Test Scores to Rank Schools," American Economic Review, American Economic Association, vol. 95(4), pages 1237-1258, September.
    3. Eric A. Hanushek & Margaret E. Raymond, 2005. "Does school accountability lead to improved student performance?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 24(2), pages 297-327.
    4. Cecilia Elena Rouse & Jane Hannaway & Dan Goldhaber & David Figlio, 2013. "Feeling the Florida Heat? How Low-Performing Schools Respond to Voucher and Accountability Pressure," American Economic Journal: Economic Policy, American Economic Association, vol. 5(2), pages 251-281, May.
    5. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja, 2017. "Report Cards: The Impact of Providing School and Child Test Scores on Educational Markets," American Economic Review, American Economic Association, vol. 107(6), pages 1535-1563, June.
    6. Marco Manacorda, 2012. "The Cost of Grade Retention," The Review of Economics and Statistics, MIT Press, vol. 94(2), pages 596-606, May.
    7. Susanna Loeb & Katharine Strunk, 2007. "Accountability and Local Control: Response to Incentives with and without Authority over Resource Generation and Allocation," Education Finance and Policy, MIT Press, vol. 2(1), pages 10-39, January.
    8. Jonah Rockoff & Lesley J. Turner, 2010. "Short-Run Impacts of Accountability on School Quality," American Economic Journal: Economic Policy, American Economic Association, vol. 2(4), pages 119-147, November.
    9. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    10. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    11. Thomas Ahn & Jacob Vigdor, 2014. "The Impact of No Child Left Behind's Accountability Sanctions on School Performance: Regression Discontinuity Evidence from North Carolina," NBER Working Papers 20511, National Bureau of Economic Research, Inc.
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    14. Mizala, Alejandra & Urquiola, Miguel, 2013. "School markets: The impact of information approximating schools' effectiveness," Journal of Development Economics, Elsevier, vol. 103(C), pages 313-335.
    15. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    16. Hanley Chiang, "undated". "How Accountability Pressure on Failing Schools Affects Student Achievement," Mathematica Policy Research Reports c58a3b537e324447b94a2bd41, Mathematica Policy Research.
    17. Karthik Muralidharan & Venkatesh Sundararaman, 2010. "The Impact of Diagnostic Feedback to Teachers on Student Learning: Experimental Evidence from India," Economic Journal, Royal Economic Society, vol. 120(546), pages 187-203, August.
    18. Mizala, Alejandra & Romaguera, Pilar & Urquiola, Miguel, 2007. "Socioeconomic status or noise? Tradeoffs in the generation of school quality information," Journal of Development Economics, Elsevier, vol. 84(1), pages 61-75, September.
    19. John Deke & Lisa Dragoset, "undated". "Statistical Power for Regression Discontinuity Designs in Education: Empirical Estimates of Design Effects Relative to Randomized Controlled Trials," Mathematica Policy Research Reports a4f1d03eb7bf427a8983d4736, Mathematica Policy Research.
    20. repec:mpr:mprres:7486 is not listed on IDEAS
    21. David N. Figlio & Lawrence S. Getzler, 2002. "Accountability , Ability and Disability: Gaming the System," NBER Working Papers 9307, National Bureau of Economic Research, Inc.
    22. repec:mpr:mprres:6364 is not listed on IDEAS
    23. Pierre Koning & Karen van der Wiel, 2013. "Ranking The Schools: How School-Quality Information Affects School Choice In The Netherlands," Journal of the European Economic Association, European Economic Association, vol. 11(2), pages 466-493, April.
    24. Barbara Bruns & Deon Filmer & Harry Anthony Patrinos, 2011. "Making Schools Work : New Evidence on Accountability Reforms," World Bank Publications - Books, The World Bank Group, number 2270, April.
    25. Peter Z. Schochet, "undated". "Statistical Power for Random Assignment Evaluations of Education Programs," Mathematica Policy Research Reports 6749d31ad72d4acf988f7dce5, Mathematica Policy Research.
    26. Chiang, Hanley, 2009. "How accountability pressure on failing schools affects student achievement," Journal of Public Economics, Elsevier, vol. 93(9-10), pages 1045-1057, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Daniela Scur & Raffaella Sadun & John Van Reenen & Renata Lemos & Nicholas Bloom, 2021. "The World Management Survey at 18: lessons and the way forward," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 37(2), pages 231-258.
    2. Machado, Anaely & Terra, Rafael & Tannuri-Pianto, Maria, 2024. "Higher education responses to accountability," Economics of Education Review, Elsevier, vol. 98(C).
    3. De Hoyos Navarro,Rafael E. & Estrada,Ricardo & Vargas Mancera,Maria Jose, 2021. "Do Large-Scale Student Assessments Really Capture Cognitive Skills ?," Policy Research Working Paper Series 9537, The World Bank.
    4. de Hoyos, Rafael & Estrada, Ricardo & Vargas, María José, 2021. "What do test scores really capture? Evidence from a large-scale student assessment in Mexico," World Development, Elsevier, vol. 146(C).
    5. World Bank Group, 2015. "Slovak Republic Skilling Up the Next Generation," World Bank Publications - Reports 23330, The World Bank Group.
    6. de Hoyos, Rafael & Djaker, Sharnic & Ganimian, Alejandro J. & Holland, Peter A., 2024. "The impact of combining performance-management tools and training with diagnostic feedback in public schools: Experimental evidence from Argentina," Economics of Education Review, Elsevier, vol. 99(C).
    7. Lee Crawfurd, 2017. "School Management and Public–Private Partnerships in Uganda," Journal of African Economies, Centre for the Study of African Economies, vol. 26(5), pages 539-560.
    8. Chicas Romero,Mauricio & Bedoya,Juan & Yanez Pagans,Monica & Silveyra De La Garza,Marcela Lucia & De Hoyos Navarro,Rafael E., 2021. "School Management, Grants, and Test Scores : Experimental Evidence from Mexico," Policy Research Working Paper Series 9535, The World Bank.
    9. Djaker, Sharnic & Ganimian, Alejandro J. & Sabarwal, Shwetlena, 2024. "Out of sight, out of mind? The gap between students’ test performance and teachers’ estimations in India and Bangladesh," Economics of Education Review, Elsevier, vol. 102(C).
    10. Padilla-Romo, María, 2022. "Full-time schools, policy-induced school switching, and academic performance," Journal of Economic Behavior & Organization, Elsevier, vol. 196(C), pages 79-103.
    11. Romero, Mauricio & Bedoya, Juan & Yanez-Pagans, Monica & Silveyra, Marcela & de Hoyos, Rafael, 2022. "Direct vs indirect management training: Experimental evidence from schools in Mexico," Journal of Development Economics, Elsevier, vol. 154(C).
    12. Mauricio Romero & Juan Bedoya & Monica Yanez‐Pagans & Marcela Silveyra & Rafael de Hoyos, 2024. "The effect of school grants on test scores: experimental evidence from Mexico," Economica, London School of Economics and Political Science, vol. 91(363), pages 980-995, July.
    13. Jere R. Behrman & Susan W. Parker & Petra Todd & Weilong Zhang, 2025. "Prospering through Prospera: A dynamic model of CCT impacts on educational attainment and achievement in Mexico," Quantitative Economics, Econometric Society, vol. 16(1), pages 133-183, January.
    14. Berry, James & Kannan, Harini & Mukherji, Shobhini & Shotland, Marc, 2020. "Failure of frequent assessment: An evaluation of India’s continuous and comprehensive evaluation program," Journal of Development Economics, Elsevier, vol. 143(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Feng, Li & Figlio, David & Sass, Tim, 2018. "School accountability and teacher mobility," Journal of Urban Economics, Elsevier, vol. 103(C), pages 1-17.
    2. Michael Coelli & Gigi Foster & Andrew Leigh, 2018. "Do School Principals Respond to Increased Public Scrutiny? New Survey Evidence from Australia," The Economic Record, The Economic Society of Australia, vol. 94(S1), pages 73-101, June.
    3. Nunes, Luis C. & Reis, Ana Balcão & Seabra, Carmo, 2015. "The publication of school rankings: A step toward increased accountability?," Economics of Education Review, Elsevier, vol. 49(C), pages 15-23.
    4. Morozumi, Atsuyoshi & Tanaka, Ryuichi, 2020. "Should School-Level Results of National Assessments Be Made Public?," IZA Discussion Papers 13450, Institute of Labor Economics (IZA).
    5. Pierre Koning & Karen Wiel, 2012. "School Responsiveness to Quality Rankings: An Empirical Analysis of Secondary Education in the Netherlands," De Economist, Springer, vol. 160(4), pages 339-355, December.
    6. Cuesta, José Ignacio & González, Felipe & Larroulet Philippi, Cristian, 2020. "Distorted quality signals in school markets," Journal of Development Economics, Elsevier, vol. 147(C).
    7. Hemelt, Steven W., 2011. "Performance effects of failure to make Adequate Yearly Progress (AYP): Evidence from a regression discontinuity framework," Economics of Education Review, Elsevier, vol. 30(4), pages 702-723, August.
    8. Holbein, John B. & Ladd, Helen F., 2017. "Accountability pressure: Regression discontinuity estimates of how No Child Left Behind influenced student behavior," Economics of Education Review, Elsevier, vol. 58(C), pages 55-67.
    9. Woo, Seokjin & Lee, Soohyung & Kim, Kyunghee, 2015. "Carrot and stick?: Impact of a low-stakes school accountability program on student achievement," Economics Letters, Elsevier, vol. 137(C), pages 195-199.
    10. Chiang, Hanley, 2009. "How accountability pressure on failing schools affects student achievement," Journal of Public Economics, Elsevier, vol. 93(9-10), pages 1045-1057, October.
    11. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja, 2017. "Report Cards: The Impact of Providing School and Child Test Scores on Educational Markets," American Economic Review, American Economic Association, vol. 107(6), pages 1535-1563, June.
    12. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    13. Camargo, Braz & Camelo, Rafael & Firpo, Sergio & Ponczek, Vladimir, 2014. "Information, Market Incentives, and Student Performance," IZA Discussion Papers 7941, Institute of Labor Economics (IZA).
    14. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    15. Craig, Steven G. & Imberman, Scott A. & Perdue, Adam, 2015. "Do administrators respond to their accountability ratings? The response of school budgets to accountability grades," Economics of Education Review, Elsevier, vol. 49(C), pages 55-68.
    16. Singh, Abhijeet & Berg, Petter, 2024. "Myths of official measurement: Limits to test-based education reforms with weak governance," Journal of Public Economics, Elsevier, vol. 239(C).
    17. Friesen, Jane & Harris, Benjamin Cerf & Woodcock, Simon, 2013. "Open Enrolment and Student Achievement," CLSSRN working papers clsrn_admin-2013-46, Vancouver School of Economics, revised 22 Mar 2014.
    18. Vegas, E & Ganimian, A. J., 2013. "Theory and Evidence on Teacher Policies in Developed and Developing Countries," Working Paper 104291, Harvard University OpenScholar.
    19. Dennis Epple & Richard E. Romano & Miguel Urquiola, 2017. "School Vouchers: A Survey of the Economics Literature," Journal of Economic Literature, American Economic Association, vol. 55(2), pages 441-492, June.
    20. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ecoedu:v:58:y:2017:i:c:p:123-140. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/econedurev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.