The Effect of Survey Mode on Data Quality : Experimental Evidence from Nigeria

This paper uses a large-scale experiment in rural Nigeria to study the role of survey mode—in-person versus over the phone—in survey measurement and data quality. The experimental design isolates mode effects from other common sources of errors in surveys and covers 20 outcome measures across topics such as health, labor, shocks, wellbeing, and food security. The findings indicate consistent mode effects across outcomes, with phone responses differing from in-person responses by 17-18 percent at the median. These effects are large relative to other errors in phone surveys, such as under-coverage of households without phones. A within-respondent design enables capturing the full, respondent-level distribution of mode effects and finds them to vary much more than the averages reveal. Respondents with higher education levels are less prone to mode effects, whereas mode effects sharply increase in prevalence as respondents face more answer options. As the reliance on phone surveys in low- and middle-income countries grows, these findings indicate areas with large potential for data quality gains and have first-order implications for economic research in low- and middle-income countries.