Selection Bias in College Admissions Test Scores

Data from the two leading college admissions tests—the SAT and the ACT—can provide a valuable measures of student achievement, but bias due to the non-representativeness of test takers is an important concern. We take advantage of a policy reform in Illinois that made the ACT a graduation requirement to identify the within- and across-school selectivity of ACT takers. In contrast to cross-sectional or time-differenced estimates, estimates based on the Illinois policy change indicate substantial positive selection into test participation both across and within schools. Despite this, school-level averages of observed scores are extremely highly correlated with average latent scores, as noise introduced by across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between school variation in average achievement but is otherwise unlikely to lead to misleading conclusions.