The Anatomy and Evolution of Survey Error

Surveys provide much of social science knowledge and are the source of information on private behaviors, attitudes and expectations. The literature documents four error sources: coverage, unit non-response, item non-response, and measurement error. The relative severity of error sources is difficult to assess, limiting efforts to improve accuracy and interpret survey statistics. We unify the fragmented literature by analyzing and decomposing survey error in an empirical Total Survey Error framework, quantifying every error component on a common scale of bias and absolute error. We apply our approach to 18 income and program receipt variables from the CPS using linked administrative records over two decades. For nearly half of the variables, bias exceeds 40% and eliminating all individual-level error would require changing more than 80% of the true total. The dominant source of bias is measurement error among respondents. The largest potential gains therefore come from improving reporting accuracy not raising response rates. Moreover, even when bias is stable, absolute error has tended to rise over time. Our results clarify which sources of bias require corrections and where survey design improvements could be most consequential.