IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v175y2012i4p915-938.html
   My bibliography  Save this article

Non-response biases in surveys of schoolchildren: the case of the English Programme for International Student Assessment (PISA) samples

Author

Listed:
  • John Micklewright
  • Sylke V. Schnepf
  • Chris Skinner

Abstract

We analyse response patterns to an important survey of schoolchildren, exploiting rich auxiliary information on respondents' and non-respondents' cognitive ability that is correlated both with response and the learning achievement that the survey aims to measure. The survey is the Programme for International Student Assessment (PISA), which sets response thresholds in an attempt to control the quality of data. We analyse the case of England for 2000, when response rates were deemed sufficiently high by the organizers of the survey to publish the results, and 2003, when response rates were a little lower and deemed of sufficient concern for the results not to be published. We construct weights that account for the pattern of non-response by using two methods: propensity scores and the generalized regression estimator. There is clear evidence of biases, but there is no indication that the slightly higher response rates in 2000 were associated with higher quality data. This underlines the danger of using response rate thresholds as a guide to quality of data.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • John Micklewright & Sylke V. Schnepf & Chris Skinner, 2012. "Non-response biases in surveys of schoolchildren: the case of the English Programme for International Student Assessment (PISA) samples," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(4), pages 915-938, October.
  • Handle: RePEc:bla:jorssa:v:175:y:2012:i:4:p:915-938 DOI: j.1467-985X.2012.01036.x
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/j.1467-985X.2012.01036.x
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Leopold Simar & Paul Wilson, 2000. "A general methodology for bootstrapping in non-parametric frontier models," Journal of Applied Statistics, Taylor & Francis Journals, vol. 27(6), pages 779-802.
    2. Banker, Rajiv D. & Gadh, Vandana M. & Gorr, Wilpen L., 1993. "A Monte Carlo comparison of two production frontier estimation methods: Corrected ordinary least squares and data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 67(3), pages 332-343, June.
    3. Gong, Byeong-Ho & Sickles, Robin C., 1992. "Finite sample evidence on the performance of stochastic frontiers and data envelopment analysis using panel data," Journal of Econometrics, Elsevier, vol. 51(1-2), pages 259-284.
    4. Peter C. Smith & Andrew Street, 2005. "Measuring the efficiency of public services: the limits of analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 401-417.
    5. M. Stone, 2002. "How not to measure the efficiency of public services (and how one might)," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 165(3), pages 405-434.
    6. Léopold Simar & Paul W. Wilson, 1998. "Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models," Management Science, INFORMS, vol. 44(1), pages 49-61, January.
    7. Catherine J. Morrison Paul & Warren E. Johnston & Gerald A. G. Frengley, 2000. "Efficiency in New Zealand Sheep and Beef Farming: The Impacts of Regulatory Reform," The Review of Economics and Statistics, MIT Press, vol. 82(2), pages 325-337, May.
    8. Schmidt, Peter & Lin, Tsai-Fen, 1984. "Simple tests of alternative specifications in stochastic frontier models," Journal of Econometrics, Elsevier, vol. 24(3), pages 349-361, March.
    9. Berndt, Ernst R. & Morrison, Catherine J., 1995. "High-tech capital formation and economic performance in U.S. manufacturing industries An exploratory analysis," Journal of Econometrics, Elsevier, vol. 65(1), pages 9-43, January.
    10. Kneip, Alois & Simar, Léopold & Wilson, Paul W., 2008. "Asymptotics And Consistent Bootstraps For Dea Estimators In Nonparametric Frontier Models," Econometric Theory, Cambridge University Press, vol. 24(06), pages 1663-1697, December.
    11. Daniel J. Henderson & R. Robert Russell, 2005. "Human Capital And Convergence: A Production-Frontier Approach ," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(4), pages 1167-1205, November.
    12. Fan, Yanqin & Li, Qi & Weersink, Alfons, 1996. "Semiparametric Estimation of Stochastic Production Frontier Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 14(4), pages 460-468, October.
    13. Guilkey, David K & Lovell, C A Knox & Sickles, Robin C, 1983. "A Comparison of the Performance of Three Flexible Functional Forms," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 24(3), pages 591-616, October.
    14. Jondrow, James & Knox Lovell, C. A. & Materov, Ivan S. & Schmidt, Peter, 1982. "On the estimation of technical inefficiency in the stochastic frontier production function model," Journal of Econometrics, Elsevier, vol. 19(2-3), pages 233-238, August.
    15. William C. Horrace & Peter Schmidt, 2002. "Confidence Statements for Efficiency Estimates from Stochastic Frontier Models," Econometrics 0206006, EconWPA.
    16. Bojani, Antonio N. & Caudill, Steven B. & Ford, Jon M., 1998. "Small-sample properties of ML, COLS, and DEA estimators of frontier models in the presence of heteroscedasticity," European Journal of Operational Research, Elsevier, vol. 108(1), pages 140-148, July.
    17. Park, Soo-Uk & Lesourd, Jean-Baptiste, 2000. "The efficiency of conventional fuel power plants in South Korea: A comparison of parametric and non-parametric approaches," International Journal of Production Economics, Elsevier, vol. 63(1), pages 59-67, January.
    18. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    19. Subal C. Kumbhakar & Efthymios G. Tsionas, 2011. "Stochastic error specification in primal and dual production systems," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 26(2), pages 270-297, March.
    20. KNEIP, Alois & SIMAR, Léopold, 1995. "A General Framework for Frontier Estimation with Panel Data," CORE Discussion Papers 1995060, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).
    21. Hayfield, Tristen & Racine, Jeffrey S., 2008. "Nonparametric Econometrics: The np Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 27(i05).
    22. Louis Amato & Christie Amato, 2000. "The Impact of High Tech Production Techniques on Productivity and Profitability in Selected U.S. Manufacturing Industries," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 16(4), pages 327-342, June.
    23. Dassler, Thoralf & Parker, David & Saal, David S., 2006. "Methods and trends of performance benchmarking in UK utility regulation," Utilities Policy, Elsevier, vol. 14(3), pages 166-174, September.
    24. Aigner, Dennis & Lovell, C. A. Knox & Schmidt, Peter, 1977. "Formulation and estimation of stochastic frontier production function models," Journal of Econometrics, Elsevier, vol. 6(1), pages 21-37, July.
    25. Cubbin, John & Tzanidakis, George, 1998. "Regression versus data envelopment analysis for efficiency measurement: an application to the England and Wales regulated water industry," Utilities Policy, Elsevier, vol. 7(2), pages 75-85, June.
    26. Wilson, Paul W., 2008. "FEAR: A software package for frontier efficiency analysis with R," Socio-Economic Planning Sciences, Elsevier, vol. 42(4), pages 247-254, December.
    27. Cubbin, John, 2005. "Efficiency in the water industry," Utilities Policy, Elsevier, vol. 13(4), pages 289-293, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jakubowski, Maciej & Pokropek, Artur, 2015. "Reading achievement progress across countries," International Journal of Educational Development, Elsevier, vol. 45(C), pages 77-88.
    2. Brick J. Michael, 2013. "Unit Nonresponse and Weighting Adjustments: A Critical Review," Journal of Official Statistics, De Gruyter Open, vol. 29(3), pages 329-353, June.
    3. John Jerrim & Anna Vignoles & Ross Finnie, 2012. "University access for disadvantaged children: A comparison across English speaking countries," DoQSS Working Papers 12-11, Department of Quantitative Social Science - UCL Institute of Education, University College London.
    4. Daniel K. Lew & Amber Himes-Cornell & Jean Lee, 2015. "Weighting and Imputation for Missing Data in a Cost and Earnings Fishery Survey," Marine Resource Economics, University of Chicago Press, vol. 30(2), pages 219-230.
    5. Micklewright, John & Schnepf, Sylke V. & Silva, Pedro N., 2012. "Peer effects and measurement error: The impact of sampling variation in school survey data (evidence from PISA)," Economics of Education Review, Elsevier, vol. 31(6), pages 1136-1142.

    More about this item

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:175:y:2012:i:4:p:915-938. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing) or (Christopher F. Baum). General contact details of provider: http://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.