IDEAS home Printed from https://ideas.repec.org/a/vrs/offsta/v33y2017i3p801-833n11.html
   My bibliography  Save this article

Using Prior Wave Information and Paradata: Can They Help to Predict Response Outcomes and Call Sequence Length in a Longitudinal Study?

Author

Listed:
  • Durrant Gabriele B.

    (Department of Social Statistics and Demography and ESRC National Centre for Research Methods (NCRM), School of Social Sciences, University of Southampton, SO17 1BJ, Southampton, United Kingdom of Great Britain and Northern Ireland.)

  • Maslovskaya Olga

    (ESRC National Centre for Research Methods (NCRM), School of Social Sciences, University of Southampton, United Kingdom of Great Britain and Northern Ireland.)

  • Smith Peter W. F.

    (Department of Social Statistics and Demography and ESRC Administrative Data Research Centre for England (ADRC-E), School of Social Sciences, University of Southampton, United Kingdom of Great Britain and Northern Ireland.)

Abstract

In recent years the use of paradata for nonresponse investigations has risen significantly. One key question is how useful paradata, including call record data and interviewer observations, from the current and previous waves of a longitudinal study, as well as previous wave survey information, are in predicting response outcomes in a longitudinal context. This article aims to address this question. Final response outcomes and sequence length (the number of calls/visits to a household) are modelled both separately and jointly for a longitudinal study. Being able to predict length of call sequence and response can help to improve both adaptive and responsive survey designs and to increase efficiency and effectiveness of call scheduling. The article also identifies the impact of different methodological specifications of the models, for example different specifications of the response outcomes. Latent class analysis is used as one of the approaches to summarise call outcomes in sequences. To assess and compare the models in their ability to predict, indicators derived from classification tables, ROC (Receiver Operating Characteristic) curves, discrimination and prediction are proposed in addition to the standard approach of using the pseudo R2 value, which is not a sufficient indicator on its own. The study uses data from Understanding Society, a large-scale longitudinal survey in the UK. The findings indicate that basic models (including geographic, design and survey data from the previous wave), although commonly used in predicting and adjusting for nonresponse, do not predict the response outcome well. Conditioning on previous wave paradata, including call record data, interviewer observation data and indicators of change, improve the fit of the models slightly. A significant improvement can be observed when conditioning on the most recent call outcome, which may indicate that the nonresponse process predominantly depends on the most current circumstances of a sample unit.

Suggested Citation

  • Durrant Gabriele B. & Maslovskaya Olga & Smith Peter W. F., 2017. "Using Prior Wave Information and Paradata: Can They Help to Predict Response Outcomes and Call Sequence Length in a Longitudinal Study?," Journal of Official Statistics, Sciendo, vol. 33(3), pages 801-833, September.
    • Handle: RePEc:vrs:offsta:v:33:y:2017:i:3:p:801-833:n:11
    DOI: 10.1515/jos-2017-0037
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jos-2017-0037
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jos-2017-0037?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gabriele B. Durrant & Julia D'Arrigo & Fiona Steele, 2011. "Using paradata to predict best times of contact, conditioning on household and interviewer influences," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(4), pages 1029-1049, October.
    2. Lagorio, Carlos, 2016. "Call and response: modelling longitudinal contact and cooperation using Wave 1 call records data," Understanding Society Working Paper Series 2016-01, Understanding Society at the Institute for Social and Economic Research.
    3. White,Halbert, 1996. "Estimation, Inference and Specification Analysis," Cambridge Books, Cambridge University Press, number 9780521574464, October.
    4. F. Kreuter & K. Olson & J. Wagner & T. Yan & T. M. Ezzati‐Rice & C. Casas‐Cordero & M. Lemay & A. Peytchev & R. M. Groves & T. E. Raghunathan, 2010. "Using proxy measures and other correlates of survey outcomes to adjust for non‐response: examples from multiple surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(2), pages 389-407, April.
    5. Gabriele B. Durrant & Fiona Steele, 2009. "Multilevel modelling of refusal and non‐contact in household surveys: evidence from six UK Government surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(2), pages 361-381, April.
    6. Hamparsum Bozdogan, 1987. "Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions," Psychometrika, Springer;The Psychometric Society, vol. 52(3), pages 345-370, September.
    7. Sigrid Haunberger, 2010. "The effects of interviewer, respondent and area characteristics on cooperation in panel surveys: a multilevel approach," Quality & Quantity: International Journal of Methodology, Springer, vol. 44(5), pages 957-969, August.
    8. Durrant, Gabriele B. & D'Arrigo, Julia & Steele, Fiona, 2011. "Using field process data to predict best times of contact conditioning on household and interviewer influences," LSE Research Online Documents on Economics 52201, London School of Economics and Political Science, LSE Library.
    9. Gabriele B. Durrant & Julia D'Arrigo & Fiona Steele, 2013. "Analysing interviewer call record data by using a multilevel discrete time event history modelling approach," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 251-269, January.
    10. Robert M. Groves & Steven G. Heeringa, 2006. "Responsive design for household surveys: tools for actively controlling survey errors and costs," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 439-457, July.
    11. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    12. Brady T. West, 2013. "An examination of the quality and utility of interviewer observations in the National Survey of Family Growth," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 211-225, January.
    13. Paul P. Biemer & Patrick Chen & Kevin Wang, 2013. "Using level-of-effort paradata in non-response adjustments with application to field surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(1), pages 147-168, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wagner James & West Brady T. & Elliott Michael R. & Coffey Stephanie, 2020. "Comparing the Ability of Regression Modeling and Bayesian Additive Regression Trees to Predict Costs in a Responsive Survey Design Context," Journal of Official Statistics, Sciendo, vol. 36(4), pages 907-931, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
    2. Ronald R. Rindfuss & Minja K. Choe & Noriko O. Tsuya & Larry L. Bumpass & Emi Tamaki, 2015. "Do low survey response rates bias results? Evidence from Japan," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 32(26), pages 797-828.
    3. Gabriele B. Durrant & Julia D’Arrigo, 2014. "Doorstep Interactions and Interviewer Effects on the Process Leading to Cooperation or Refusal," Sociological Methods & Research, , vol. 43(3), pages 490-518, August.
    4. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    5. Lagorio, Carlos, 2016. "Call and response: modelling longitudinal contact and cooperation using Wave 1 call records data," Understanding Society Working Paper Series 2016-01, Understanding Society at the Institute for Social and Economic Research.
    6. Steele, Fiona & Durrant, Gabriele B., 2011. "Alternative approaches to multilevel modelling of survey non-contact and refusal," LSE Research Online Documents on Economics 50113, London School of Economics and Political Science, LSE Library.
    7. Hanly, Mark & Clarke, Paul & Steele, Fiona, 2016. "Sequence analysis of call record data: exploring the role of different cost settings," LSE Research Online Documents on Economics 64896, London School of Economics and Political Science, LSE Library.
    8. Plewis Ian & Shlomo Natalie, 2017. "Using Response Propensity Models to Improve the Quality of Response Data in Longitudinal Studies," Journal of Official Statistics, Sciendo, vol. 33(3), pages 753-779, September.
    9. Walejko Gina & Wagner James, 2018. "A Study of Interviewer Compliance in 2013 and 2014 Census Test Adaptive Designs," Journal of Official Statistics, Sciendo, vol. 34(3), pages 649-670, September.
    10. Jamie C. Moore & Gabriele B. Durrant & Peter W. F. Smith, 2021. "Do coefficients of variation of response propensities approximate non‐response biases during survey data collection?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 301-323, January.
    11. Kristen Olson, 2013. "Paradata for Nonresponse Adjustment," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 142-170, January.
    12. Ashmead Robert & Slud Eric & Hughes Todd, 2017. "Adaptive Intervention Methodology for Reduction of Respondent Contact Burden in the American Community Survey," Journal of Official Statistics, Sciendo, vol. 33(4), pages 901-919, December.
    13. Mark Hanly & Paul Clarke & Fiona Steele, 2016. "Sequence analysis of call record data: exploring the role of different cost settings," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(3), pages 793-808, June.
    14. Arie Preminger & Uri Ben-zion & David Wettstein, 2007. "The extended switching regression model: allowing for multiple latent state variables," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 26(7), pages 457-473.
    15. Andy Peytchev, 2013. "Consequences of Survey Nonresponse," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 88-111, January.
    16. Rebecca Vassallo & Gabriele Durrant & Peter Smith, 2017. "Separating interviewer and area effects by using a cross-classified multilevel logistic model: simulation findings and implications for survey designs," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(2), pages 531-550, February.
    17. White, Halbert & Pettenuzzo, Davide, 2014. "Granger causality, exogeneity, cointegration, and economic policy analysis," Journal of Econometrics, Elsevier, vol. 178(P2), pages 316-330.
    18. Wagner James & Olson Kristen, 2018. "An Analysis of Interviewer Travel and Field Outcomes in Two Field Surveys," Journal of Official Statistics, Sciendo, vol. 34(1), pages 211-237, March.
    19. Durrant, Gabriele B. & D'Arrigo, Julia & Steele, Fiona, 2011. "Using field process data to predict best times of contact conditioning on household and interviewer influences," LSE Research Online Documents on Economics 52201, London School of Economics and Political Science, LSE Library.
    20. Frauke Kreuter, 2013. "Facing the Nonresponse Challenge," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 23-35, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vrs:offsta:v:33:y:2017:i:3:p:801-833:n:11. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.sciendo.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.