IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v181y2018i4p1057-1075.html
   My bibliography  Save this article

Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample

Author

Listed:
  • Gabriele B. Durrant
  • Sylke V. Schnepf

Abstract

Non‐response is a major problem facing research in the social sciences including in education surveys. Hence, research is needed to understand non‐response patterns better as well as non‐response as a social phenomenon. Findings may contribute to improvements in the future designs of such surveys. Using logistic and multilevel logistic modelling we examine correlates of non‐response at the school and pupil level in the important educational achievement survey the ‘Programme for International Student Assessment’ (PISA) for England. The analysis exploits unusually rich auxiliary information on all schools and pupils sampled for PISA, whether responding or not, from two large‐scale administrative sources on pupils’ socio‐economic background and results in national public examinations. This information correlates highly with the PISA target variable. Findings show that characteristics that are associated with non‐response differ between the school and pupil levels. Our results also indicate that schools matter in explaining pupil level response, which is often ignored in non‐response analysis. Our findings have important implications for future education surveys. For example, if replacement schools are used to improve response, our results suggest that it may be more important to match initial and replacement schools on the socio‐economic composition of their pupils than on any of the factors that are currently used.

Suggested Citation

  • Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
  • Handle: RePEc:bla:jorssa:v:181:y:2018:i:4:p:1057-1075
    DOI: 10.1111/rssa.12337
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12337
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssa.12337?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Durrant, Gabriele B. & D'Arrigo, Julia & Steele, Fiona, 2011. "Using field process data to predict best times of contact conditioning on household and interviewer influences," LSE Research Online Documents on Economics 52201, London School of Economics and Political Science, LSE Library.
    2. Jon Rasbash & George Leckie & Rebecca Pillinger & Jennifer Jenkins, 2010. "Children's educational progress: partitioning family, school and area effects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(3), pages 657-682, July.
    3. W. J. Browne & S. V. Subramanian & K. Jones & H. Goldstein, 2005. "Variance partitioning in multilevel logistic models that exhibit overdispersion," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(3), pages 599-613, July.
    4. John Micklewright & Sylke V. Schnepf & Chris Skinner, 2012. "Non-response biases in surveys of schoolchildren: the case of the English Programme for International Student Assessment (PISA) samples," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(4), pages 915-938, October.
    5. George Leckie & Harvey Goldstein, 2011. "Understanding Uncertainty in School League Tables," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 207-224, June.
    6. Jan Pickery & Geert Loosveldt & Ann Carton, 2001. "The Effects of Interviewer and Respondent Characteristics on Response Behavior in Panel Surveys," Sociological Methods & Research, , vol. 29(4), pages 509-523, May.
    7. Gabriele B. Durrant & Julia D'Arrigo & Fiona Steele, 2011. "Using paradata to predict best times of contact, conditioning on household and interviewer influences," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(4), pages 1029-1049, October.
    8. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    9. Lorraine Dearden & Alfonso Miranda & Sophia Rabe‐Hesketh, 2011. "Measuring School Value Added with Administrative Data: The Problem of Missing Variables," Fiscal Studies, Institute for Fiscal Studies, vol. 32(2), pages 263-278, June.
    10. F. Kreuter & K. Olson & J. Wagner & T. Yan & T. M. Ezzati‐Rice & C. Casas‐Cordero & M. Lemay & A. Peytchev & R. M. Groves & T. E. Raghunathan, 2010. "Using proxy measures and other correlates of survey outcomes to adjust for non‐response: examples from multiple surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(2), pages 389-407, April.
    11. Durrant, Gabriele B. & Steele, Fiona, 2009. "Multilevel modelling of refusal and non-contact in household surveys: evidence from six UK Government surveys," LSE Research Online Documents on Economics 50112, London School of Economics and Political Science, LSE Library.
    12. Gabriele B. Durrant & Fiona Steele, 2009. "Multilevel modelling of refusal and non‐contact in household surveys: evidence from six UK Government surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(2), pages 361-381, April.
    13. repec:mpr:mprres:4780 is not listed on IDEAS
    14. Sophia Rabe-Hesketh & Anders Skrondal, 2012. "Multilevel and Longitudinal Modeling Using Stata, 3rd Edition," Stata Press books, StataCorp LP, edition 3, number mimus2, March.
    15. Denise Hawkes & Ian Plewis, 2006. "Modelling non‐response in the National Child Development Study," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 479-491, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schnepf, Sylke V. & Durrant, Gabriele B. & Micklewright, John, 2014. "Which Schools and Pupils Respond to Educational Achievement Surveys? A Focus on the English PISA Sample," IZA Discussion Papers 8411, Institute of Labor Economics (IZA).
    2. Durrant Gabriele B. & Maslovskaya Olga & Smith Peter W. F., 2017. "Using Prior Wave Information and Paradata: Can They Help to Predict Response Outcomes and Call Sequence Length in a Longitudinal Study?," Journal of Official Statistics, Sciendo, vol. 33(3), pages 801-833, September.
    3. Steele, Fiona & Durrant, Gabriele B., 2011. "Alternative approaches to multilevel modelling of survey non-contact and refusal," LSE Research Online Documents on Economics 50113, London School of Economics and Political Science, LSE Library.
    4. Rebecca Vassallo & Gabriele Durrant & Peter Smith, 2017. "Separating interviewer and area effects by using a cross-classified multilevel logistic model: simulation findings and implications for survey designs," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(2), pages 531-550, February.
    5. Juan Merlo & Philippe Wagner & Nermin Ghith & George Leckie, 2016. "An Original Stepwise Multilevel Logistic Regression Analysis of Discriminatory Accuracy: The Case of Neighbourhoods and Health," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-31, April.
    6. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    7. Adriana Ana Maria Davidescu & Monica Roman & Vasile Alecsandru Strat & Mihaela Mosora, 2019. "Regional Sustainability, Individual Expectations and Work Motivation: A Multilevel Analysis," Sustainability, MDPI, Open Access Journal, vol. 11(12), pages 1-23, June.
    8. Plewis Ian & Shlomo Natalie, 2017. "Using Response Propensity Models to Improve the Quality of Response Data in Longitudinal Studies," Journal of Official Statistics, Sciendo, vol. 33(3), pages 753-779, September.
    9. Michele Lalla & Maddalena Cavicchioli, 2020. "Nonresponse and measurement errors in income: matching individual survey data with administrative tax data," Department of Economics 0170, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    10. Mark Amos, 2018. "Interviewer effects on patterns of nonresponse: Evaluating the impact on the reasons for contraceptive nonuse in the Indonesia and the Philippines DHS," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 39(14), pages 415-430.
    11. Lagorio, Carlos, 2016. "Call and response: modelling longitudinal contact and cooperation using Wave 1 call records data," Understanding Society Working Paper Series 2016-01, Understanding Society at the Institute for Social and Economic Research.
    12. Martin Halla & Friedrich Schneider & Alexander Wagner, 2013. "Satisfaction with democracy and collective action problems: the case of the environment," Public Choice, Springer, vol. 155(1), pages 109-137, April.
    13. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for 'Progress 8' raised by COVID-19," CEPEO Working Paper Series 21-04, Centre for Education Policy and Equalising Opportunities, UCL Institute of Education, revised Apr 2021.
    14. Lucy Prior & John Jerrim & Dave Thomson & George Leckie, 2021. "A review and evaluation of secondary school accountability in England: Statistical strengths, weaknesses, and challenges for ‘Progress 8’ raised by COVID-19," DoQSS Working Papers 21-12, Quantitative Social Science - UCL Social Research Institute, University College London.
    15. Geert Loosveldt & Koen Beullens, 2014. "A Procedure to Assess Interviewer Effects on Nonresponse Bias," SAGE Open, , vol. 4(1), pages 21582440145, February.
    16. Barbara Felderer & Jannis Kueck & Martin Spindler, 2021. "Big Data meets Causal Survey Research: Understanding Nonresponse in the Recruitment of a Mixed-mode Online Panel," Papers 2102.08994, arXiv.org.
    17. Brick J. Michael, 2013. "Unit Nonresponse and Weighting Adjustments: A Critical Review," Journal of Official Statistics, Sciendo, vol. 29(3), pages 329-353, June.
    18. Øystein Gjerde & Kjell Knivsflå & Frode Sættem, 2010. "Evidence on competitive advantage and superior stock market performance," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 31(4), pages 277-301.
    19. Ronald R. Rindfuss & Minja K. Choe & Noriko O. Tsuya & Larry L. Bumpass & Emi Tamaki, 2015. "Do low survey response rates bias results? Evidence from Japan," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 32(26), pages 797-828.
    20. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:181:y:2018:i:4:p:1057-1075. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.