The Implications of “Contamination†for Experimental Design in Education

Experimental designs that randomly assign entire clusters of individuals (e.g., schools and classrooms) to treatments are frequently advocated as a way of guarding against contamination of the estimated average causal effect of treatment. However, in the absence of contamination, experimental designs that randomly assign intact clusters to treatments are less efficient than designs that randomly assign individual units within clusters. The current article considers the case of contamination processes that tend to make experimental and control subjects appear more similar than they truly are. The article demonstrates that, for most parameter values of practical interest, the statistical power of a randomized block (RB) design remains higher than the power of a cluster randomized (CR) design even when contamination causes the effect size to decrease by as much as 10%–60%. Furthermore, from the standpoint of point estimation, RB designs will tend to be preferred when true effect sizes are small and when the number of clusters in the experiment is not too large, but CR designs will tend to be preferred when true effect sizes are large or when the number of clusters in the experiment is large.