IDEAS home Printed from https://ideas.repec.org/p/ucl/cepeow/21-04.html
   My bibliography  Save this paper

A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8' raised by COVID-19

Author

Listed:
  • Lucy Prior

    (Centre for Multilevel Modelling, School of Education, University of Bristol)

  • John Jerrim

    (UCL Social Research Institute, University College London)

  • Dave Thomson

    (FFT Education Datalab)

  • George Leckie

    (Centre for Multilevel Modelling, School of Education, University of Bristol)

Abstract

School performance measures are published annually in England to hold schools to account and to support parental school choice. This article reviews and evaluates the 'Progress 8' secondary school accountability system for state-funded schools. We assess the statistical strengths and weaknesses of Progress 8 relating to: choice of pupil outcome attainment measure; potential adjustments for pupil input attainment and background characteristics; decisions around which schools and pupils are excluded from the measure; presentation of Progress 8 to users, choice of statistical model, and calculation of statistical uncertainty; and issues related to the volatility of school performance over time, including scope for reporting multi-year averages. We then discuss challenges for Progress 8 raised by the COVID-19 pandemic. Six simple recommendations follow to improve Progress 8 and school accountability in England.

Suggested Citation

  • Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8' raised by COVID-19," CEPEO Working Paper Series 21-04, Centre for Education Policy and Equalising Opportunities, UCL Institute of Education, revised Apr 2021.
  • Handle: RePEc:ucl:cepeow:21-04
    as

    Download full text from publisher

    File URL: https://repec-cepeo.ucl.ac.uk/cepeow/cepeowp21-04.pdf
    File Function: First version, 2021
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. George Leckie & Harvey Goldstein, 2011. "Understanding Uncertainty in School League Tables," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 207-224, June.
    2. George Leckie, 2009. "The complexity of school and neighbourhood effects and movements of pupils on school differences in models of educational achievement," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 537-554, June.
    3. John Jerrim, 2020. "Measuring socio-economic background using administrative data. What is the best proxy available?," DoQSS Working Papers 20-09, Quantitative Social Science - UCL Social Research Institute, University College London.
    4. Harvey Goldstein & Michael J. R. Healy, 1995. "The Graphical Presentation of a Collection of Means," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(1), pages 175-177, January.
    5. Lorraine Dearden & Alfonso Miranda & Sophia Rabe‐Hesketh, 2011. "Measuring School Value Added with Administrative Data: The Problem of Missing Variables," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 263-278, June.
    6. Rebecca Allen & Simon Burgess, 2011. "Can School League Tables Help Parents Choose Schools?," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 245-261, June.
    7. J Taylor, 2011. "The English Baccalaureate: how not to measure school performance," Working Papers 616395, Lancaster University Management School, Economics Department.
    8. Deborah Wilson, 2004. "Which Ranking? The Impact of a 'Value-Added' Measure of Secondary School Performance," Public Money & Management, Taylor & Francis Journals, vol. 24(1), pages 37-45.
    9. Katherine E. Castellano & Sophia Rabe-Hesketh & Anders Skrondal, 2014. "Composition, Context, and Endogeneity in School and Teacher Comparisons," Journal of Educational and Behavioral Statistics, , vol. 39(5), pages 333-367, October.
    10. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for ‘Progress 8’ raised by COVID-19," DoQSS Working Papers 21-12, Quantitative Social Science - UCL Social Research Institute, University College London.
    2. Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
    3. Allen, Rebecca & Burgess, Simon, 2013. "Evaluating the provision of school performance information for school choice," Economics of Education Review, Elsevier, vol. 34(C), pages 175-190.
    4. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    5. Isabella Sulis & Mariano Porcu & Vincenza Capursi, 2019. "On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(3), pages 1305-1331, April.
    6. Juan Merlo & Philippe Wagner & Nermin Ghith & George Leckie, 2016. "An Original Stepwise Multilevel Logistic Regression Analysis of Discriminatory Accuracy: The Case of Neighbourhoods and Health," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    7. Schnepf, Sylke V. & Durrant, Gabriele B. & Micklewright, John, 2014. "Which Schools and Pupils Respond to Educational Achievement Surveys? A Focus on the English PISA Sample," IZA Discussion Papers 8411, Institute of Labor Economics (IZA).
    8. Leonardo Grilli & Carla Rampichini, 2015. "Specification of random effects in multilevel models: a review," Quality & Quantity: International Journal of Methodology, Springer, vol. 49(3), pages 967-976, May.
    9. Jon Rasbash & George Leckie & Rebecca Pillinger & Jennifer Jenkins, 2010. "Children's educational progress: partitioning family, school and area effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 657-682, July.
    10. Columbu, Silvia & Porcu, Mariano & Sulis, Isabella, 2021. "University choice and the attractiveness of the study area: Insights on the differences amongst degree programmes in Italy based on generalised mixed-effect models," Socio-Economic Planning Sciences, Elsevier, vol. 74(C).
    11. Helen Simpson, 2009. "Productivity In Public Services," Journal of Economic Surveys, Wiley Blackwell, vol. 23(2), pages 250-276, April.
    12. Arpino, Bruno & Varriale, Roberta, 2009. "Assessing the quality of institutions’ rankings obtained through multilevel linear regression models," MPRA Paper 19873, University Library of Munich, Germany.
    13. Pickery, Jan, 2002. "Contextual effects on the vote in Germany: A multilevel analysis," Discussion Papers, Research Unit: Institutions and Social Change FS III 02-202, WZB Berlin Social Science Center.
    14. Claudia Herresthal, 2015. "Inferring School Quality from Rankings: The Impact of School Choice," Economics Series Working Papers 747, University of Oxford, Department of Economics.
    15. Sulis, Isabella & Giambona, Francesca & Porcu, Mariano, 2020. "Adjusted indicators of quality and equity for monitoring the education systems over time. Insights on EU15 countries from PISA surveys," Socio-Economic Planning Sciences, Elsevier, vol. 69(C).
    16. Simon Burgess & Carol Propper & Helen Slater & Deborah Wilson, 2005. "Who wins and who loses from school accountability? The distribution of educational gain in English secondary schools," The Centre for Market and Public Organisation 05/128, The Centre for Market and Public Organisation, University of Bristol, UK.
    17. Simon Burgess & Brendon McConnell & Carol Propper & Deborah Wilson, 2004. "Sorting and Choice in English Secondary Schools," The Centre for Market and Public Organisation 04/111, The Centre for Market and Public Organisation, University of Bristol, UK.
    18. Bosker, R.J. & van de Loo, P.J.E. & van der Velden, R.K.W., 1997. "Differential effects of colleges on the labour market success of their graduates," ROA Research Memorandum 1E, Maastricht University, Research Centre for Education and the Labour Market (ROA).
    19. Isabella Sulis & Mariano Porcu, 2012. "Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 21(2), pages 193-209, June.
    20. Deborah Wilson & Bronwyn Croxson & Adele Atkinson, 2004. "“What Gets Measured Gets Done”: Headteachers’ Responses to the English Secondary School," The Centre for Market and Public Organisation 04/107, The Centre for Market and Public Organisation, University of Bristol, UK.

    More about this item

    Keywords

    Progress 8; school performance measures; school accountability; school choice; school league tables; value-added model; COVID-19;
    All these keywords.

    JEL classification:

    • I2 - Health, Education, and Welfare - - Education

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ucl:cepeow:21-04. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/epucluk.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Jake Anders (email available below). General contact details of provider: https://edirc.repec.org/data/epucluk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.