IDEAS home Printed from https://ideas.repec.org/a/eee/pubeco/v106y2013icp57-67.html
   My bibliography  Save this article

A natural experiment in school accountability: The impact of school performance information on pupil progress

Author

Listed:
  • Burgess, Simon
  • Wilson, Deborah
  • Worth, Jack

Abstract

We test the hypothesis that the abolition of school performance tables negatively affects school effectiveness. Our data allow us to implement a classic difference-in-difference analysis comparing outcomes in England and Wales, before and after the abolition of the tables in Wales. We find significant and robust evidence that this reform markedly reduced school effectiveness in Wales relative to England. There is significant heterogeneity across schools: schools in the top quartile of the league tables show no effect. We also report that the reform had no systematic significant impact on either sorting by ability or by socioeconomic status.

Suggested Citation

  • Burgess, Simon & Wilson, Deborah & Worth, Jack, 2013. "A natural experiment in school accountability: The impact of school performance information on pupil progress," Journal of Public Economics, Elsevier, vol. 106(C), pages 57-67.
    • Handle: RePEc:eee:pubeco:v:106:y:2013:i:c:p:57-67
    DOI: 10.1016/j.jpubeco.2013.06.005
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0047272713001291
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jpubeco.2013.06.005?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    2. Stephen G. Donald & Kevin Lang, 2007. "Inference with Difference-in-Differences and Other Panel Data," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 221-233, May.
    3. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    4. Machin, Stephen & McNally, Sandra, 2008. "The literacy hour," Journal of Public Economics, Elsevier, vol. 92(5-6), pages 1441-1462, June.
    5. Simon Burgess & Ellen Greaves & Anna Vignoles & Deborah Wilson, 2015. "What Parents Want: School Preferences and School Choice," Economic Journal, Royal Economic Society, vol. 125(587), pages 1262-1289, September.
    6. Reback, Randall, 2008. "Teaching to the rating: School accountability and the distribution of student achievement," Journal of Public Economics, Elsevier, vol. 92(5-6), pages 1394-1415, June.
    7. Lechner, Michael, 2011. "The Estimation of Causal Effects by Difference-in-Difference Methods," Foundations and Trends(R) in Econometrics, now publishers, vol. 4(3), pages 165-224, November.
    8. Derek Neal & Diane Whitmore Schanzenbach, 2010. "Left Behind by Design: Proficiency Counts and Test-Based Accountability," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 263-283, May.
    9. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    10. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 533-575.
    11. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    12. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    13. Propper Carol & Sutton Matt & Whitnall Carolyn & Windmeijer Frank, 2008. "Did 'Targets and Terror' Reduce Waiting Times in England for Hospital Care?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(2), pages 1-27, January.
    14. Helen F. Ladd & Edward B. Fiske, 2008. "Handbook of Research in Education Finance and Policy," Education Finance and Policy, MIT Press, vol. 3(1), pages 149-150, January.
    15. Brian A. Jacob & Steven D. Levitt, 2003. "Rotten Apples: An Investigation of the Prevalence and Predictors of Teacher Cheating," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 118(3), pages 843-877.
    16. Thomas S. Dee & Brian Jacob, 2011. "The impact of no Child Left Behind on student achievement," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 30(3), pages 418-446, June.
    17. Eric A. Hanushek & Margaret E. Raymond, 2005. "Does school accountability lead to improved student performance?," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 24(2), pages 297-327.
    18. Hansen, Christian B., 2007. "Asymptotic properties of a robust variance matrix estimator for panel data when T is large," Journal of Econometrics, Elsevier, vol. 141(2), pages 597-620, December.
    19. Simon Burgess & Carol Propper & Helen Slater & Deborah Wilson, 2005. "Who wins and who loses from school accountability? The distribution of educational gain in English secondary schools," The Centre for Market and Public Organisation 05/128, The Centre for Market and Public Organisation, University of Bristol, UK.
    20. Justine S. Hastings & Jeffrey M. Weinstein, 2008. "Information, School Choice, and Academic Achievement: Evidence from Two Experiments," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 123(4), pages 1373-1414.
    21. Richard Blundell & Monica Costa Dias & Costas Meghir & John Van Reenen, 2004. "Evaluating the Employment Impact of a Mandatory Job Search Program," Journal of the European Economic Association, MIT Press, vol. 2(4), pages 569-606, June.
    22. Mizala, Alejandra & Urquiola, Miguel, 2013. "School markets: The impact of information approximating schools' effectiveness," Journal of Development Economics, Elsevier, vol. 103(C), pages 313-335.
    23. Gwyn Bevan & Richard Hamblin, 2009. "Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 161-190, January.
    24. Alejandra Mizala & Miguel Urquiola, 2007. "Parental choice and school markets: The impact of information approximating school effectiveness," Documentos de Trabajo 239, Centro de Economía Aplicada, Universidad de Chile.
    25. Propper, Carol & Sutton, Matt & Whitnall, Carolyn & Windmeijer, Frank, 2010. "Incentives and targets in hospital care: Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 94(3-4), pages 318-335, April.
    26. Moulton, Brent R, 1990. "An Illustration of a Pitfall in Estimating the Effects of Aggregate Variables on Micro Unit," The Review of Economics and Statistics, MIT Press, vol. 72(2), pages 334-338, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. MOROZUMI Atsuyoshi & TANAKA Ryuichi, 2023. "School Accountability and Student Achievement: Neighboring schools matter," Discussion papers 23004, Research Institute of Economy, Trade and Industry (RIETI).
    2. Annika B. Bergbauer & Eric A. Hanushek & Ludger Woessmann, 2024. "Testing," Journal of Human Resources, University of Wisconsin Press, vol. 59(2), pages 349-388.
    3. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    4. Colin Green & Ole Henning Nyhus & Kari Vea Salvanes, "undated". "How does testing young children influence educational attainment and well-being?," Working Paper Series 19422, Department of Economics, Norwegian University of Science and Technology.
    5. Burgess, Simon, 2016. "Human Capital and Education: The State of the Art in the Economics of Education," IZA Discussion Papers 9885, Institute of Labor Economics (IZA).
    6. Beland, Louis-Philippe & Murphy, Richard, 2016. "Ill Communication: Technology, distraction & student performance," Labour Economics, Elsevier, vol. 41(C), pages 61-76.
    7. Santosh Kumar & Ramona Molitor & Sebastian Vollmer, 2014. "Children of Drought: Rainfall Shocks and Early Child Health in Rural India," Working Papers 1407, Sam Houston State University, Department of Economics and International Business.
    8. Gjefsen, Hege Marie & Gunnes, Trude, 2016. "The effects of School Accountability on Teacher Mobility and Teacher Sorting," MPRA Paper 69664, University Library of Munich, Germany.
    9. Gerdes, Christer, 2015. "Does Performance Information Affect Job Seekers in Selecting Private Providers in Voucher-Based ALMP Programs?," IZA Discussion Papers 8992, Institute of Labor Economics (IZA).
    10. Carmen Aina & Massimiliano Bratti & Enrico Lippo, 2021. "Ranking high schools using university student performance in Italy," Economia Politica: Journal of Analytical and Institutional Economics, Springer;Fondazione Edison, vol. 38(1), pages 293-321, April.
    11. Kim, Dongwoo & Koedel, Cory & Ni, Shawn & Podgursky, Michael, 2017. "Labor market frictions and production efficiency in public schools," Economics of Education Review, Elsevier, vol. 60(C), pages 54-67.
    12. Hege Marie Gjefsen & Trude Gunnes, 2015. "School accountability Incentives or sorting?," Discussion Papers 815, Statistics Norway, Research Department.
    13. Joseph Regan-Stansfield, 2017. "The Impact of Standardised Testing on Later High Stakes Test Outcomes," Working Papers 161114030, Lancaster University Management School, Economics Department.
    14. Ellen Greaves & Iftikhar Hussain & Birgitta Rabe & Imran Rasul, 2023. "Parental Responses to Information about School Quality: Evidence from Linked Survey and Administrative Data," The Economic Journal, Royal Economic Society, vol. 133(654), pages 2334-2402.
    15. Ruth Lupton & Polina Obolenskaya, 2013. "Labour's Record on Education: Policy, Spending and Outcomes 1997-2010," CASE - Social Policy in a Cold Climate Working Paper 03, Centre for Analysis of Social Exclusion, LSE.
    16. Morozumi, Atsuyoshi & Tanaka, Ryuichi, 2020. "Should School-Level Results of National Assessments Be Made Public?," IZA Discussion Papers 13450, Institute of Labor Economics (IZA).
    17. Yuta Kuroda, 2022. "What does the disclosure of school quality information bring? The effect through the housing market," Journal of Regional Science, Wiley Blackwell, vol. 62(1), pages 125-149, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    3. Roel Elk & Marc Steeg & Dinand Webbink, 2013. "Can Financial Incentives for Regional Education Authorities Reduce School Dropout?," De Economist, Springer, vol. 161(4), pages 367-398, December.
    4. Haeck, Catherine & Lefebvre, Pierre & Merrigan, Philip, 2014. "The distributional impacts of a universal school reform on mathematical achievements: A natural experiment from Canada," Economics of Education Review, Elsevier, vol. 41(C), pages 137-160.
    5. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    6. Figlio, D. & Karbownik, K. & Salvanes, K.G., 2016. "Education Research and Administrative Data," Handbook of the Economics of Education,, Elsevier.
    7. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 318, University of California, Davis, Department of Economics.
    8. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
    9. Seth Gershenson & Stephen B. Holt & Nicholas Papageorge, 2015. "Who Believes in Me? The Effect of Student-Teacher Demographic Match on Teacher Expectations," Upjohn Working Papers 15-231, W.E. Upjohn Institute for Employment Research.
    10. De Witte, Kristof & Geys, Benny & Solondz, Catharina, 2014. "Public expenditures, educational outcomes and grade inflation: Theory and evidence from a policy intervention in the Netherlands," Economics of Education Review, Elsevier, vol. 40(C), pages 152-166.
    11. Morozumi, Atsuyoshi & Tanaka, Ryuichi, 2020. "Should School-Level Results of National Assessments Be Made Public?," IZA Discussion Papers 13450, Institute of Labor Economics (IZA).
    12. Richardson, J.T., 2015. "Accountability incentives and academic achievement: Distributional impacts of accountability when standards are set low," Economics of Education Review, Elsevier, vol. 44(C), pages 1-16.
    13. A. Colin Cameron & Douglas L. Miller, 2010. "Robust Inference with Clustered Data," Working Papers 106, University of California, Davis, Department of Economics.
    14. Evan Riehl & Meredith Welch, 2023. "Accountability, Test Prep Incentives, and the Design of Math and English Exams," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 60-96, January.
    15. Cain Polidano & Justin Ven & Sarah Voitchovsky, 2021. "Are Broad-Based Vouchers an Effective Way to Support Life-Long Learning? Evidence from an Australian Reform," Research in Higher Education, Springer;Association for Institutional Research, vol. 62(7), pages 998-1038, November.
    16. Verzulli, R.; & Lippi Bruni, M.;, 2022. "The quicker the better: Fostering timely responses in public hospitals," Health, Econometrics and Data Group (HEDG) Working Papers 22/08, HEDG, c/o Department of Economics, University of York.
    17. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    18. Sansone, Dario, 2019. "Pink work: Same-sex marriage, employment and discrimination," Journal of Public Economics, Elsevier, vol. 180(C).
    19. Seth Gershenson, 2016. "Performance Standards and Employee Effort: Evidence From Teacher Absences," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 35(3), pages 615-638, June.
    20. Eduardo Fé & Bruce Hollingsworth, 2016. "Short- and long-run estimates of the local effects of retirement on health," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(4), pages 1051-1067, October.

    More about this item

    Keywords

    School accountability; School effectiveness; Performance tables;
    All these keywords.

    JEL classification:

    • I20 - Health, Education, and Welfare - - Education - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:pubeco:v:106:y:2013:i:c:p:57-67. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/inca/505578 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.