Quasi-Experimental Evaluation of Alternative Sample Selection Corrections

Researchers routinely use datasets where outcomes of interest are unobserved for some cases, potentially creating a sample selection problem. Statisticians and econometricians have proposed many selection correction methods to address this challenge. We use a natural experiment to evaluate different sample selection correction methods’ performance. From 2007, the state of Michigan required that all students take a college entrance exam, increasing the exam-taking rate from 64% to 99% and largely eliminating selection into exam-taking. We apply different selection correction methods, using different sets of covariates, to the selected exam score data from before 2007. We compare the estimated coefficients from the selection-corrected models to those from OLS regressions using the complete exam score data from after 2007 as a benchmark. We find that less restrictive semiparametric correction methods typically perform better than parametric correction methods but not better than simple OLS regressions that do not correct for selection. Performance is generally worse for models that use only a few discrete covariates than for models that use more covariates with less coarse distributions.