IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/21399.html
   My bibliography  Save this paper

Household Surveys in Crisis

Author

Listed:
  • Bruce D. Meyer
  • Wallace K.C. Mok
  • James X. Sullivan

Abstract

Household surveys, one of the main innovations in social science research of the last century, are threatened by declining accuracy due to reduced cooperation of respondents. While many indicators of survey quality have steadily declined in recent decades, the literature has largely emphasized rising nonresponse rates rather than other potentially more important dimensions to the problem. We divide the problem into rising rates of nonresponse, imputation, and measurement error, documenting the rise in each of these threats to survey quality over the past three decades. A fundamental problem in assessing biases due to these problems in surveys is the lack of a benchmark or measure of truth, leading us to focus on the accuracy of the reporting of government transfers. We provide evidence from aggregate measures of transfer reporting as well as linked microdata. We discuss the relative importance of misreporting of program receipt and conditional amounts of benefits received, as well as some of the conjectured reasons for declining cooperation and survey errors. We end by discussing ways to reduce the impact of the problem including the increased use of administrative data and the possibilities for combining administrative and survey data.

Suggested Citation

  • Bruce D. Meyer & Wallace K.C. Mok & James X. Sullivan, 2015. "Household Surveys in Crisis," NBER Working Papers 21399, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:21399
    Note: AG CH HC HE LS PE
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w21399.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lisa Barrow & Jonathan Davis, 2012. "The upside of down: postsecondary enrollment in the Great Recession," Economic Perspectives, Federal Reserve Bank of Chicago, vol. 36(Q IV), pages 117-129.
    2. John M. Abowd & Martha H. Stinson, 2013. "Estimating Measurement Error in Annual Job Earnings: A Comparison of Survey and Administrative Data," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1451-1467, December.
    3. Adam Bee & Bruce D. Meyer & James X. Sullivan, 2013. "The Validity of Consumption Data: Are the Consumer Expenditure Interview and Diary Surveys Informative?," NBER Chapters, in: Improving the Measurement of Consumer Expenditures, pages 204-240, National Bureau of Economic Research, Inc.
    4. Christopher R. Bollinger & Barry T. Hirsch, 2006. "Match Bias from Earnings Imputation in the Current Population Survey: The Case of Imperfect Matching," Journal of Labor Economics, University of Chicago Press, vol. 24(3), pages 483-520, July.
    5. Bollinger, Christopher R & David, Martin H, 2001. "Estimation with Response Error and Nonresponse: Food-Stamp Participation in the SIPP," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(2), pages 129-141, April.
    6. Bruce D. Meyer & Wallace K. C. Mok & James X. Sullivan, 2009. "The Under-Reporting of Transfers in Household Surveys: Its Nature and Consequences," NBER Working Papers 15181, National Bureau of Economic Research, Inc.
    7. Mark Aguiar & Erik Hurst, 2007. "Measuring Trends in Leisure: The Allocation of Time Over Five Decades," The Quarterly Journal of Economics, Oxford University Press, vol. 122(3), pages 969-1006.
    8. Scherpf, Erik & Newman, Constance & Prell, Mark, 2014. "Targeting of Supplemental Nutrition Assistance Program Benefits: Evidence from the ACS and NY SNAP Administrative Records," 2014 Annual Meeting, July 27-29, 2014, Minneapolis, Minnesota 174295, Agricultural and Applied Economics Association.
    9. Duncan, Greg J & Hill, Daniel H, 1989. "Assessing the Quality of Household Panel Data: The Case of the Panel Study of Income Dynamics," Journal of Business & Economic Statistics, American Statistical Association, vol. 7(4), pages 441-452, October.
    10. Rebecca R. Andridge & Roderick J. A. Little, 2010. "A Review of Hot Deck Imputation for Survey Non‐response," International Statistical Review, International Statistical Institute, vol. 78(1), pages 40-64, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bruce D. Meyer & Nikolas Mittag, 2015. "Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net," NBER Working Papers 21676, National Bureau of Economic Research, Inc.
    2. Mittag, Nikolas, 2016. "Correcting for Misreporting of Government Benefits," IZA Discussion Papers 10266, Institute of Labor Economics (IZA).
    3. Meyer, Bruce D. & Mittag, Nikolas, 2019. "Combining Administrative and Survey Data to Improve Income Measurement," IZA Discussion Papers 12266, Institute of Labor Economics (IZA).
    4. Bruckmeier, Kerstin & Müller, Gerrit & Riphahn, Regina T., 2015. "Survey misreporting of welfare receipt—Respondent, interviewer, and interview characteristics," Economics Letters, Elsevier, vol. 129(C), pages 103-107.
    5. Bruce D. Meyer & Wallace K. C. Mok & James X. Sullivan, 2009. "The Under-Reporting of Transfers in Household Surveys: Its Nature and Consequences," NBER Working Papers 15181, National Bureau of Economic Research, Inc.
    6. Bruce Meyer & Nikolas Mittag, 2013. "Misclassification In Binary Choice Models," Working Papers 13-27, Center for Economic Studies, U.S. Census Bureau.
    7. Meyer, Bruce D. & Mittag, Nikolas, 2017. "Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net," IZA Discussion Papers 10943, Institute of Labor Economics (IZA).
    8. Todd, Jessica E. & Newman, Constance & Ver Ploeg, Michele, 2010. "Changing Participation in Food Assistance Programs Among Low-Income Children After Welfare Reform," Economic Research Report 58613, United States Department of Agriculture, Economic Research Service.
    9. Melvin Stephens & Takashi Unayama, 2019. "Estimating the Impacts of Program Benefits: Using Instrumental Variables with Underreported and Imputed Data," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 468-475, July.
    10. Bruce D. Meyer & Nikolas Mittag, 2019. "Combining Administrative and Survey Data to Improve Income Measurement," NBER Working Papers 25738, National Bureau of Economic Research, Inc.
    11. Bollinger, Christopher R. & Hirsch, Barry & Hokayem, Charles M. & Ziliak, James P., 2018. "Trouble in the Tails? What We Know about Earnings Nonresponse Thirty Years after Lillard, Smith, and Welch," IZA Discussion Papers 11710, Institute of Labor Economics (IZA).
    12. Molly Dahl & Thomas DeLeire & Shannon Mok, 2012. "Food Insufficiency and Income Volatility in U.S. Households: The Effects of Imputed Earnings in the Survey of Income and Program Participation: Working Paper 2012-07," Working Papers 43137, Congressional Budget Office.
    13. Bruce Meyer & Wallace K. C. Mok, 2016. "Disability, Earnings, Income and Consumption," NBER Chapters, in: Social Insurance Programs (Trans-Atlantic Public Economics Seminar, TAPES), National Bureau of Economic Research, Inc.
    14. Bruce Meyer & Nikolas Mittag, 2017. "Using Linked Survey and Administrative Data to Better Measure Income: Implications for Poverty, Program Effectiveness and Holes in the Safety Net," Working Papers 2017-075, Human Capital and Economic Opportunity Working Group.
    15. Kerstin Bruckmeier & Gerrit M�ller & Regina T. Riphahn, 2014. "Who misreports welfare receipt in surveys?," Applied Economics Letters, Taylor & Francis Journals, vol. 21(12), pages 812-816, August.
    16. Bruce Meyer & Robert Goerge, 2011. "Errors in Survey Reporting and Imputation and Their Effects on Estimates of Food Stamp Program Participation," Working Papers 11-14, Center for Economic Studies, U.S. Census Bureau.
    17. Raymundo M. Campos-Vázquez, 2013. "Efectos de los ingresos no reportados en el nivel y tendencia de la pobreza laboral en México," Ensayos Revista de Economia, Universidad Autonoma de Nuevo Leon, Facultad de Economia, vol. 0(2), pages 23-54, November.
    18. Kerstin Bruckmeier & Katrin Hohmeyer & Stefan Schwarz, 2018. "Welfare receipt misreporting in survey data and its consequences for state dependence estimates: new insights from linked administrative and survey data," Journal for Labour Market Research, Springer;Institute for Employment Research/ Institut für Arbeitsmarkt- und Berufsforschung (IAB), vol. 52(1), pages 1-21, December.
    19. Nguimkeu, Pierre & Denteh, Augustine & Tchernis, Rusty, 2019. "On the estimation of treatment effects with endogenous misreporting," Journal of Econometrics, Elsevier, vol. 208(2), pages 487-506.
    20. Keller, Elisa, 2019. "Labor supply and gender differences in occupational choice," European Economic Review, Elsevier, vol. 115(C), pages 221-241.

    More about this item

    JEL classification:

    • C42 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Survey Methods
    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • H53 - Public Economics - - National Government Expenditures and Related Policies - - - Government Expenditures and Welfare Programs
    • H55 - Public Economics - - National Government Expenditures and Related Policies - - - Social Security and Public Pensions
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty
    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:21399. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (). General contact details of provider: http://edirc.repec.org/data/nberrus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.