Can The Continuum of Resistance Model Be Applied to Understand Mode Selection in Sequential Mixed-Mode Surveys?

Background: Analyses of sequential mixed-mode survey data can be biased due to non-random selection of participants into mode. Understanding the consequence of this bias requires knowledge of the predictors of mode selection. However, research on these predictors is sparse, in contrast with research on the predictors of survey non-response. The ‘continuum of resistance’ model of survey response predicts that delayed responders – who, in sequential mixed-mode surveys, appear in later offered modes – and non-responders share similar characteristics. If the model, which is testable in longitudinal data, is correct, this would suggest that research on non-response could be generalized to understand mode selection. Methods: We used data from a major UK birth cohort study (the 1958 National Child Development Study) which embedded a sequential web-then-telephone mixed-mode survey at the age 55y sweep to assess whether (a) descriptively, late (i.e. telephone) and non-responders share similar characteristics and (b) whether predictions from models of non-response are accurate when used to predict telephone response. For (a), we calculated univariate descriptive statistics and performed cluster analysis to compare an array of participant characteristics across response groups (web, telephone and non-response). For (b), we estimated random forest models for non-response and telephone response (conditional on response) and compared two metrics of predictive accuracy (Area Under the Receiver Operating Characteristic curve [AUC ROC] and Brier scores) when using the non-response model to instead predict telephone response, against predictions from the model generated for telephone response, specifically. Results: Telephone and non-respondents were similar on almost all (measured) characteristics, and dissimilar in most regards to web respondents. Predictions from non-response models had similar predictive accuracy to predictions from models trained on telephone response, specifically – for instance, AUC ROC values in hold-out samples not used to train models of 0.72 (95% CI = 0.70, 0.74) and 0.74 (95% CI = 0.72, 0.75), respectively. Conclusions: The characteristics of late- and non-responders in a sequential (web-then-telephone) mixed-mode survey were very similar, consistent with the ‘continuum of resistance’ model of survey response. This suggests that research on non-response could transport to understanding mode selection in sequential mixed-mode surveys, though replications in other surveys with different mixed-mode designs (e.g., modes adopted and their order) is required.