Measuring and testing the interview mode effect in mixed mode surveys

Many studies are showing an increased tendency to use more than one data collection mode for a particular survey. However, mixed data collection modes may influence responses given by interviewees and require researchers to verify if differences in responses, when present, are ascribable to the type of data collection mode. Often, random assignment is not feasible and requires researchers to solve an additional and not negligible problem, namely to verify if differences in responses are ascribable to the self selection or to the type of data collection mode being used. The aim of the present paper is to measure the mode effect on the answers using a new data driven multivariate approach, that allows to disentangle the interview mode effect on answers from the effect of self selection. We will work through the use of the new multivariate method with AlmaLaurea case concerning the evaluation of two different data collection methods, the CAWI (Computer Assisted Web Interviewing) and the CATI (Computer Assisted Telephone Interviewing). As with any new statistical method, the success of this method depends on its efficacy in relation to that of the existing methods. Therefore, results of the multivariate approach will be compared to the Propensity Score method that AlmaLaurea usually applies to identify the presence of an interview mode effect. Both methods produce similar results.