IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/22333.html
   My bibliography  Save this paper

Difficulty to Reach Respondents and Nonresponse Bias: Evidence from Large Government Surveys

Author

Listed:
  • Ori Heffetz
  • Daniel B. Reeves

Abstract

How high is unemployment? How low is labor force participation? Is obesity more prevalent among men? How large are household expenditures? We study the sources of the relevant official statistics—the Current Population Survey (CPS), the Behavioral Risk Factor Surveillance System (BRFSS), and the Consumer Expenditure Survey (CEX)—and find that the answers depend on whether we look at easy- or at difficult-to-reach respondents, measured by the number of call and visit attempts made by interviewers. A challenge to the (conditionally-)random-nonresponse assumption, these findings empirically substantiate the theoretical warning against making population-wide estimates from surveys with low response rates.

Suggested Citation

  • Ori Heffetz & Daniel B. Reeves, 2016. "Difficulty to Reach Respondents and Nonresponse Bias: Evidence from Large Government Surveys," NBER Working Papers 22333, National Bureau of Economic Research, Inc.
    • Handle: RePEc:nbr:nberwo:22333
    Note: AG LS
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w22333.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alan B. Krueger & Alexandre Mas & Xiaotong Niu, 2017. "The Evolution of Rotation Group Bias: Will the Real Unemployment Rate Please Stand Up?," The Review of Economics and Statistics, MIT Press, vol. 99(2), pages 258-264, May.
    2. Adrian Chadi, 2019. "Dissatisfied with life or with being interviewed? Happiness and the motivation to participate in a survey," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 53(3), pages 519-553, October.
    3. Luc Behaghel & Bruno Crépon & Marc Gurgand & Thomas Le Barbanchon, 2015. "Please Call Again: Correcting Nonresponse Bias in Treatment Effect Models," The Review of Economics and Statistics, MIT Press, vol. 97(5), pages 1070-1080, December.
    4. Adrian Chadi & Clemens Hetschko, 2018. "The magic of the new: How job changes affect job satisfaction," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 27(1), pages 23-39, March.
    5. Paul P. Biemer & Patrick Chen & Kevin Wang, 2013. "Using level-of-effort paradata in non-response adjustments with application to field surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 147-168, January.
    6. Richard Williams, 2011. "Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects," CHI11 Stata Conference 9, Stata Users Group.
    7. Ori Heffetz & Matthew Rabin, 2013. "Conclusions Regarding Cross-Group Differences in Happiness Depend on Difficulty of Reaching Respondents," American Economic Review, American Economic Association, vol. 103(7), pages 3001-3021, December.
    8. Oswald, Andrew J. & Wu, Stephen, 2010. "Objective Confirmation of Subjective Measures of Human Well-being: Evidence from the USA," IZA Discussion Papers 4695, Institute of Labor Economics (IZA).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Albert Cheng & Gema Zamarro & Bart Orriens, 2020. "Personality as a Predictor of Unit Nonresponse in an Internet Panel," Sociological Methods & Research, , vol. 49(3), pages 672-698, August.
    2. Warnke, Arne Jonas, 2017. "An investigation of record linkage refusal and its implications for empirical research," ZEW Discussion Papers 17-031, ZEW - Leibniz Centre for European Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Borgschulte, Mark & Cho, Heepyung & Lubotsky, Darren, 2022. "Partisanship and survey refusal," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 332-357.
    2. Makarski, Krzysztof & Tyrowicz, Joanna, 2019. "On welfare effects of increasing retirement age," Journal of Policy Modeling, Elsevier, vol. 41(4), pages 718-746.
    3. Jan-Emmanuel De Neve & George Ward & Femke De Keulenaer & Bert Van Landeghem & Georgios Kavetsos & Michael I. Norton, 2018. "The Asymmetric Experience of Positive and Negative Economic Growth: Global Evidence Using Subjective Well-Being Data," The Review of Economics and Statistics, MIT Press, vol. 100(2), pages 362-375, May.
    4. de Pinto Marco & Goerke Laszlo, 2019. "Efficiency Wages in Cournot-Oligopoly," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 19(4), pages 1-13, October.
    5. de New, Sonja C. & Schurer, Stefanie & Leung, Felix, 2015. "Testing the Validity of Item Non-Response as a Proxy for Cognitive and Non-Cognitive Skills," IZA Discussion Papers 8874, Institute of Labor Economics (IZA).
    6. Nikolova, Elena & Sanfey, Peter, 2016. "How much should we trust life satisfaction data? Evidence from the Life in Transition Survey," Journal of Comparative Economics, Elsevier, vol. 44(3), pages 720-731.
    7. Bernhardt, Robert & Wunnava, Phanindra V., 2020. "The CPS Citizenship Question and Survey Refusals: Causal and Semi-Causal Evidence Featuring a Two-Stage Regression Discontinuity Design," IZA Discussion Papers 13350, Institute of Labor Economics (IZA).
    8. Lovett, Nicholas & Xue, Yuhan, 2020. "Family first or the kindness of strangers? Foster care placements and adult outcomes," Labour Economics, Elsevier, vol. 65(C).
    9. Wu, Wenjie, 2015. "Rail access and subjective well-being: Evidence from quality of life surveys," Journal of Comparative Economics, Elsevier, vol. 43(2), pages 456-470.
    10. Sonja C. Kassenboehmer & Stefanie Schurer, 2018. "Survey item-response behavior as an imperfect proxy for unobserved ability: Theory and application," Working Papers 2018-035, Human Capital and Economic Opportunity Working Group.
    11. Van Landeghem, Bert & Vandeplas, Anneleen, 2018. "The relationship between status and happiness: Evidence from the caste system in rural India," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 77(C), pages 62-71.
    12. Lionel WILNER, 2019. "The Dynamics of Individual Happiness," Working Papers 2019-18, Center for Research in Economics and Statistics.
    13. Hajdu, Tamás & Hajdu, Gábor, 2011. "A hasznosság és a relatív jövedelem kapcsolatának vizsgálata magyar adatok segítségével [Examining the relation of utility and relative income using Hungarian data]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(1), pages 56-73.
    14. Chodorow-Reich, Gabriel & Coglianese, John, 2021. "Projecting unemployment durations: A factor-flows simulation approach with application to the COVID-19 recession," Journal of Public Economics, Elsevier, vol. 197(C).
    15. Tomas Hanell, 2022. "Unmet Aspirations and Urban Malaise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 164(1), pages 83-103, November.
    16. Everding, Jakob & Marcus, Jan, 2020. "The effect of unemployment on the smoking behavior of couples," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 29(2), pages 154-170.
    17. Susanne Väth & Michael Kirk, 2014. "Do property rights and contract farming matter for rural development? Evidence from a large-scale investment in Ghana," MAGKS Papers on Economics 201416, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    18. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    19. McGovern, Mark E. & Canning, David & Bärnighausen, Till, 2018. "Accounting for non-response bias using participation incentives and survey design: An application using gift vouchers," Economics Letters, Elsevier, vol. 171(C), pages 239-244.
    20. Dorsett, Richard & Oswald, Andrew J., 2014. "Human Well-being and In-Work Benefits: A Randomized Controlled Trial," IZA Discussion Papers 7943, Institute of Labor Economics (IZA).

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • I18 - Health, Education, and Welfare - - Health - - - Government Policy; Regulation; Public Health
    • J60 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:22333. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.