IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v175y2012i1p289-308.html
   My bibliography  Save this article

The impact of questioning method on measurement error in panel survey measures of benefit receipt: evidence from a validation study

Author

Listed:
  • Peter Lynn
  • Annette Jäckle
  • Stephen P. Jenkins
  • Emanuela Sala

Abstract

We assess measurement error in panel survey reports of social security benefit receipt, drawing on a unique validation study. Our aims are threefold. First, we quantify the incidence of measurement errors (under- and over-reporting). Second, we assess the extent to which this varies according to the questioning method that is used. Specifically, dependent interviewing has been proposed as a way to reduce under-reporting in some circumstances. We compare two versions of dependent interviewing with traditional independent interviewing in an experimental design. Third, we identify and assess new ways of reducing measurement error in panel surveys. We use data from a large-scale UK household panel survey and we consider six benefits. To assess the measurement error, a validation exercise was conducted, with administrative data on benefit receipt matched at the individual level to the survey microdata.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Peter Lynn & Annette Jäckle & Stephen P. Jenkins & Emanuela Sala, 2012. "The impact of questioning method on measurement error in panel survey measures of benefit receipt: evidence from a validation study," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(1), pages 289-308, January.
    • Handle: RePEc:bla:jorssa:v:175:y:2012:i:1:p:289-308
    DOI: j.1467-985X.2011.00717.x
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/j.1467-985X.2011.00717.x
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/j.1467-985X.2011.00717.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Bound, John & Brown, Charles & Mathiowetz, Nancy, 2001. "Measurement error in survey data," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 5, chapter 59, pages 3705-3843, Elsevier.
    2. Stephen P. Jenkins & Lorenzo Cappellari & Peter Lynn & Annette Jäckle & Emanuela Sala, 2006. "Patterns of consent: evidence from a general household survey," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 701-722, October.
    3. Jäckle, Annette, 2008. "Measurement error and data collection methods: effects on estimates from event history data," ISER Working Paper Series 2008-13, Institute for Social and Economic Research.
    4. Ruth Hancock & Geraldine Barker, 2005. "The quality of social security benefit data in the British Family Resources Survey: implications for investigating income support take‐up by pensioners," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 63-82, January.
    5. Jäckle, Annette & Noah Uhrig, S.C. & Laurie, Heather, 2007. "The introduction of dependent interviewing on the British Household Panel Survey," ISER Working Paper Series 2007-07, Institute for Social and Economic Research.
    6. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "Validation of survey data on income and employment: the ISMIE experience," ISER Working Paper Series 2004-14, Institute for Social and Economic Research.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. H. Xavier Jara & Marcelo Varela, 2017. "Tax-benefit microsimulation and income redistribution in Ecuador," WIDER Working Paper Series 177, World Institute for Development Economic Research (UNU-WIDER).
    2. Whitaker, Stephan D., 2018. "Big Data versus a survey," The Quarterly Review of Economics and Finance, Elsevier, vol. 67(C), pages 285-296.
    3. Mike Brewer & Ben Etheridge & Cormac O’Dea, 2017. "Why are Households that Report the Lowest Incomes So Well‐off?," Economic Journal, Royal Economic Society, vol. 127(605), pages 24-49, October.
    4. Lugtig Peter & Jäckle Annette, 2014. "Can I Just Check...? Effects of Edit Check Questions on Measurement Error and Survey Estimates," Journal of Official Statistics, Sciendo, vol. 30(1), pages 45-62, March.
    5. Serena Pattaro & Nick Bailey & Chris Dibben, 2020. "Using Linked Longitudinal Administrative Data to Identify Social Disadvantage," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 147(3), pages 865-895, February.
    6. Jennifer Roberts & Karl Taylor, 2022. "New Evidence on Disability Benefit Claims in Britain: The Role of Health and the Local Labour Market," Economica, London School of Economics and Political Science, vol. 89(353), pages 131-160, January.
    7. Bollinger, Christopher R. & Tasseva, Iva, 2023. "Income source confusion using the SILC," LSE Research Online Documents on Economics 119351, London School of Economics and Political Science, LSE Library.
    8. Jäckle, Annette & Johannes, Eggs, 2014. "Dependent interviewing and sub-optimal responding," ISER Working Paper Series 2014-32, Institute for Social and Economic Research.
    9. Luis Ayala & Ana Pérez & Mercedes Prieto-Alaiz, 2022. "The impact of different data sources on the level and structure of income inequality," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 13(3), pages 583-611, September.
    10. R. Bollinger, Christopher & Valentinova Tasseva, Iva, 2022. "Income source confusion using the SILC," ISER Working Paper Series 2022-04, Institute for Social and Economic Research.
    11. Paul Fisher, 2019. "Does Repeated Measurement Improve Income Data Quality?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 81(5), pages 989-1011, October.
    12. H Xavier Jara & Marcelo Varela, 2019. "Tax-benefit Microsimulation and Income Redistribution in Ecuador," International Journal of Microsimulation, International Microsimulation Association, vol. 12(1), pages 52-82.
    13. Paul Fisher & Omar Hussein, 2023. "Understanding Society: the income data," Fiscal Studies, John Wiley & Sons, vol. 44(4), pages 377-397, December.
    14. Bruckmeier, Kerstin & Riphahn, Regina T. & Wiemers, Jürgen, 2019. "Benefit underreporting in survey data and its consequences for measuring non-take-up: new evidence from linked administrative and survey data," IAB-Discussion Paper 201906, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    15. Bucks, Brian & Pence, Karen, 2015. "Wealth, pensions, debt, and savings: Considerations for a panel survey," Journal of Economic and Social Measurement, IOS Press, issue 1-4, pages 151-175.
    16. repec:esx:essedp:736 is not listed on IDEAS
    17. Tasseva, Iva Valentinova, 2016. "Evaluating the performance of means-tested benefits in Bulgaria," Journal of Comparative Economics, Elsevier, vol. 44(4), pages 919-935.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lugtig Peter & Jäckle Annette, 2014. "Can I Just Check...? Effects of Edit Check Questions on Measurement Error and Survey Estimates," Journal of Official Statistics, Sciendo, vol. 30(1), pages 45-62, March.
    2. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "The impact of interviewing method on measurement error in panel survey measures of benefit receipt: evidence from a validation study," ISER Working Paper Series 2004-28, Institute for Social and Economic Research.
    3. Bruckmeier, Kerstin & Riphahn, Regina T. & Wiemers, Jürgen, 2019. "Benefit underreporting in survey data and its consequences for measuring non-take-up: new evidence from linked administrative and survey data," IAB-Discussion Paper 201906, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    4. Korbmacher, Julie M. & Schröder, Mathis, 2013. "Consent when Linking Survey Data with Administrative Records: The Role of the Interviewer," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 7(2), pages 115-131.
    5. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "Linking household survey and administrative record data: what should the matching variables be?," ISER Working Paper Series 2004-23, Institute for Social and Economic Research.
    6. Lynn, Peter & Sala, Emanuela, 2005. "The impact of a mixed-mode data collection design on non response bias on a business survey," ISER Working Paper Series 2005-16, Institute for Social and Economic Research.
    7. Emanuela Sala & Peter Lynn, 2009. "The potential of a multi-mode data collection design to reduce non response bias. The case of a survey of employers," Quality & Quantity: International Journal of Methodology, Springer, vol. 43(1), pages 123-136, January.
    8. SC Noah Uhrig & Emanuela Sala, 2011. "When Change Matters: An Analysis of Survey Interaction in Dependent Interviewing on the British Household Panel Study," Sociological Methods & Research, , vol. 40(2), pages 333-366, May.
    9. Jennifer C. Smith, 2015. "Pay Growth, Fairness, and Job Satisfaction: Implications for Nominal and Real Wage Rigidity," Scandinavian Journal of Economics, Wiley Blackwell, vol. 117(3), pages 852-877, July.
    10. Smith, Jennifer C., 2013. "Pay Growth, Fairness and Job Satisfaction: Implications for Nominal and Real Wage Rigidity," Economic Research Papers 270540, University of Warwick - Department of Economics.
    11. Lüthen Holger & Schröder Carsten & Grabka Markus M. & Goebel Jan & Penz Hannah & Mika Tatjana & Brüggmann Daniel & Ellert Sebastian, 2022. "SOEP-RV: Linking German Socio-Economic Panel Data to Pension Records," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 242(2), pages 291-307, April.
    12. Jäckle, Annette & Lugtig, Peter, 2011. "Can I just check…? Effects of edit check questions on measurement error and survey estimates," ISER Working Paper Series 2011-23, Institute for Social and Economic Research.
    13. Sala, Emanuela & Noah Uhrig, S.C., 2009. "When change matters: the effect of dependent interviewing on survey interaction in the British Household Panel Study," ISER Working Paper Series 2009-09, Institute for Social and Economic Research.
    14. John Abowd & Martha Stinson, 2011. "Estimating Measurement Error in SIPP Annual Job Earnings: A Comparison of Census Bureau Survey and SSA Administrative Data," Working Papers 11-20, Center for Economic Studies, U.S. Census Bureau.
    15. Kaspar W thrich, 2013. "Set Identification of Generalized Linear Predictors in the Presence of Non-Classical Measurement Errors," Diskussionsschriften dp1304, Universitaet Bern, Departement Volkswirtschaft.
    16. Liran Einav & Ephraim Leibtag & Aviv Nevo, 2010. "Recording discrepancies in Nielsen Homescan data: Are they present and do they matter?," Quantitative Marketing and Economics (QME), Springer, vol. 8(2), pages 207-239, June.
    17. G. Miller & Yuriy Pylypchuk, 2014. "Marital Status, Spousal Characteristics, and the Use of Preventive Care," Journal of Family and Economic Issues, Springer, vol. 35(3), pages 323-338, September.
    18. Fabian T C Schmidt & Clemens M Lechner & Daniel Danner, 2020. "New wine in an old bottle? A facet-level perspective on the added value of Grit over BFI–2 Conscientiousness," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-25, February.
    19. Brent Kreider & Steven C. Hill, 2009. "Partially Identifying Treatment Effects with an Application to Covering the Uninsured," Journal of Human Resources, University of Wisconsin Press, vol. 44(2).
    20. Ruth Hancock & Marcello Morciano & Stephen Pudney & Francesca Zantomio, 2015. "Do household surveys give a coherent view of disability benefit targeting?: a multisurvey latent variable analysis for the older population in Great Britain," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 815-836, October.

    More about this item

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:175:y:2012:i:1:p:289-308. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.