What Can We Learn from Experimenting with Survey Methods?

This review covers a nascent literature that experiments with survey design to measure whether the way in which we collect socio-economic data in developing countries influences the data and affects the results of subsequent analyses. We start by showing that survey methods matter and the size of the survey design effects can be nothing short of staggering, affecting basic stylized facts of development (such as country rankings by poverty levels) and conclusions drawn from econometric analyses (such as what the returns to education are or whether small farm plots are more productive than large ones). We describe some of the emerging best practices for conducting survey experiments, including benchmarking against the truth, delving into the error-generating mechanisms, and documenting the costs of different survey approaches.