Estimating Measurement Error in SIPP Annual Job Earnings: A Comparison of Census Survey and SSA Administrative Data

This paper seeks to quantify sources of variation in annual job earnings data collected by the 1996 Survey of Income and Program Participation (SIPP) and to determine how much of the variation is the result of measurement error. To this end, jobs reported in the SIPP are linked to jobs reported in a new administrative database, the Detailed Earnings Records (DER) drawn from the Social Security Administration's Master Earnings File, a universe file of all earnings reported on W-2 tax forms. As a result, each job has two earnings observations per year: survey and administrative. Unlike previous validation studies, both of these earnings measures are viewed as noisy measures of some underlying true amount of annual earnings. While the existence of survey error resulting from respondent mistakes or misinterpretation is widely accepted, the idea that administrative data is also error-prone is new. However we feel that the possibility of employer reporting error, employee under-reporting of compensation such as tips, and general differences between how earnings may be reported on tax forms and in surveys, necessitates the discarding of the assumption that administrative data is a ?true? measure of the quantity collected by the survey. Exploiting the presence of individuals with multiple jobs and shared employers over time, we stack all earnings measures for each individual and estimate an econometric model that includes random person and firm effects as well as a common error component shared by SIPP and SSA earnings. In addition, the estimation equation includes two independent error components that represent the variation unique to each earnings measure. The random person and firm effects and the shared residual are interpreted as components of "true" variation that represent differences in earnings across people, firms and time periods due to underlying economic reasons. The independent error terms are interpreted as measurement error. The ratio of true variation to total variation for the SIPP earnings measure is .82, indicating that 18 percent of the variation in SIPP annual job earnings can be attributed to measurement error. In contrast the ratio of true to total variation for the DER earnings measure is .79. We also estimate a model that allows for independent AR(1) processes in all three error terms and find auto-correlation parameters of .59 for the common component, .14 for the SIPP measurement error, and .73 for the DER measurement error. These relative magnitudes imply that for first-differenced earnings, in contrast to earnings levels, the reliability ratio will be lower for the SIPP than the DER.