IDEAS home Printed from https://ideas.repec.org/a/bla/istatr/v88y2020i3p642-657.html
   My bibliography  Save this article

Is there a 'safe area' where the nonresponse rate has only a modest effect on bias despite non‐ignorable nonresponse?

Author

Listed:
  • Dan Hedlin

Abstract

Rising nonresponse rates in social surveys make the issue of nonresponse bias contentious. There are conflicting messages about the importance of high response rates and the hazards of low rates. Some articles (e.g. Groves and Peytcheva, 2008) suggest that the response rate is in general not a good predictor of survey quality. Equally, it is well known that nonresponse may induce bias and increase data collection costs. We go back in the history of the literature of nonresponse and suggest a possible reason to the notion that even a rather small nonresponse rate makes the quality of a survey debatable. We also explore the relationship between nonresponse rate and bias, assuming non‐ignorable nonresponse and focusing on estimates of totals or means. We show that there is a ‘safe area’ enclosed by the response rate on the one hand and the correlation between the response propensity and the study variable on the other hand; in this area, (1) the response rate does not greatly affect the nonresponse bias, and (2) the nonresponse bias is small.

Suggested Citation

  • Dan Hedlin, 2020. "Is there a 'safe area' where the nonresponse rate has only a modest effect on bias despite non‐ignorable nonresponse?," International Statistical Review, International Statistical Institute, vol. 88(3), pages 642-657, December.
    • Handle: RePEc:bla:istatr:v:88:y:2020:i:3:p:642-657
    DOI: 10.1111/insr.12359
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/insr.12359
    Download Restriction: no
    File URL: https://libkey.io/10.1111/insr.12359?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. John Micklewright & Sylke V. Schnepf & Chris Skinner, 2012. "Non-response biases in surveys of schoolchildren: the case of the English Programme for International Student Assessment (PISA) samples," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(4), pages 915-938, October.
    2. John L. Czajka & Amy Beyler, "undated". "Declining Response Rates in Federal Surveys: Trends and Implications (Background Paper)," Mathematica Policy Research Reports a714f76e878f4a74a6ad9f15d, Mathematica Policy Research.
    3. F. Kreuter & K. Olson & J. Wagner & T. Yan & T. M. Ezzati‐Rice & C. Casas‐Cordero & M. Lemay & A. Peytchev & R. M. Groves & T. E. Raghunathan, 2010. "Using proxy measures and other correlates of survey outcomes to adjust for non‐response: examples from multiple surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(2), pages 389-407, April.
    4. Jamie C. Moore & Gabriele B. Durrant & Peter W. F. Smith, 2018. "Data set representativeness during data collection in three UK social surveys: generalizability and the effects of auxiliary covariate choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(1), pages 229-248, January.
    5. Houshmand Shirani-Mehr & David Rothschild & Sharad Goel & Andrew Gelman, 2018. "Disentangling Bias and Variance in Election Polls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 607-614, April.
    6. Shlomo, Natalie & Skinner, Chris J. & Schouten, Barry, 2012. "Estimation of an indicator of the representativeness of survey response," LSE Research Online Documents on Economics 39124, London School of Economics and Political Science, LSE Library.
    7. J. Michael Brick & Michael E. Jones, 2008. "Propensity to respond and nonresponse bias," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(1), pages 51-73.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brick J. Michael, 2013. "Unit Nonresponse and Weighting Adjustments: A Critical Review," Journal of Official Statistics, Sciendo, vol. 29(3), pages 329-353, June.
    2. Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
    3. Barry Schouten & Fannie Cobben & Peter Lundquist & James Wagner, 2016. "Does more balanced survey response imply less non-response bias?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(3), pages 727-748, June.
    4. Roberts Caroline & Herzing Jessica M.E. & Vandenplas Caroline, 2020. "A Validation of R-Indicators as a Measure of the Risk of Bias using Data from a Nonresponse Follow-Up Survey," Journal of Official Statistics, Sciendo, vol. 36(3), pages 675-701, September.
    5. Olga Maslovskaya & Peter Lugtig, 2022. "Representativeness in six waves of CROss‐National Online Survey (CRONOS) panel," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 851-871, July.
    6. David Cutler & Kaushik Ghosh & Irina Bondarenko & Kassandra Messer & Trivellore Raghunathan & Susan Stewart & Allison B. Rosen, 2018. "Attributing Medical Spending to Conditions: A Comparison of Methods," NBER Working Papers 25233, National Bureau of Economic Research, Inc.
    7. Wu Shiya & Moerbeek Mirjam & Schouten Barry & Meijers Ralph, 2022. "Data Collection Expert Prior Elicitation in Survey Design: Two Case Studies," Journal of Official Statistics, Sciendo, vol. 38(2), pages 637-662, June.
    8. repec:cup:judgdm:v:15:y:2020:i:5:p:863-880 is not listed on IDEAS
    9. Micklewright, John & Schnepf, Sylke V. & Silva, Pedro N., 2012. "Peer effects and measurement error: The impact of sampling variation in school survey data (evidence from PISA)," Economics of Education Review, Elsevier, vol. 31(6), pages 1136-1142.
    10. Andy Peytchev, 2013. "Consequences of Survey Nonresponse," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 88-111, January.
    11. Carl-Erik Särndal & Imbi Traat & Kaur Lumiste, 2018. "Interaction Between Data Collection And Estimation Phases In Surveys With Nonresponse," Statistics in Transition New Series, Polish Statistical Association, vol. 19(2), pages 183-200, June.
    12. Ahmed, Rashad & Pesaran, M. Hashem, 2022. "Regional heterogeneity and U.S. presidential elections: Real-time 2020 forecasts and evaluation," International Journal of Forecasting, Elsevier, vol. 38(2), pages 662-687.
    13. Stan Becker & Amanda Kalamar, 2018. "Sampling Weights for Analyses of Couple Data: Example of the Demographic and Health Surveys," Demography, Springer;Population Association of America (PAA), vol. 55(4), pages 1447-1473, August.
    14. Jakubowski, Maciej & Pokropek, Artur, 2015. "Reading achievement progress across countries," International Journal of Educational Development, Elsevier, vol. 45(C), pages 77-88.
    15. Patrick Gleiser & Joseph W. Sakshaug & Marieke Volkert & Peter Ellguth & Susanne Kohaut & Iris Möller, 2022. "Introducing Web in a mixed‐mode establishment survey: Effects on nonresponse," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(3), pages 891-915, July.
    16. John Jerrim & Anna Vignoles & Ross Finnie, 2012. "University access for disadvantaged children: A comparison across English speaking countries," DoQSS Working Papers 12-11, Quantitative Social Science - UCL Social Research Institute, University College London.
    17. Durrant Gabriele B. & Maslovskaya Olga & Smith Peter W. F., 2017. "Using Prior Wave Information and Paradata: Can They Help to Predict Response Outcomes and Call Sequence Length in a Longitudinal Study?," Journal of Official Statistics, Sciendo, vol. 33(3), pages 801-833, September.
    18. Raphael Nishimura & James Wagner & Michael Elliott, 2016. "Alternative Indicators for the Risk of Non-response Bias: A Simulation Study," International Statistical Review, International Statistical Institute, vol. 84(1), pages 43-62, April.
    19. Tobias Gummer & Bella Struminskaya, 2023. "Early and Late Participation during the Field Period: Response Timing in a Mixed-Mode Probability-Based Panel Survey," Sociological Methods & Research, , vol. 52(2), pages 909-932, May.
    20. Tobias Gummer, 2019. "Assessing Trends and Decomposing Change in Nonresponse Bias: The Case of Bias in Cohort Distributions," Sociological Methods & Research, , vol. 48(1), pages 92-115, February.
    21. Frauke Kreuter & Kristen Olson, 2011. "Multiple Auxiliary Variables in Nonresponse Adjustment," Sociological Methods & Research, , vol. 40(2), pages 311-332, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:istatr:v:88:y:2020:i:3:p:642-657. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/isiiinl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.