The use of client surveys to gauge the threat of contamination in welfare reform experiments

This paper examines the type of evidence policy analysts have used to identify the presence and magnitude of contamination in welfare reform experiments. Peter Rossi's critique of the New Jersey Family Development Program evaluation motivates the following discussion. In this critique Rossi and others contend that client misperception about experimental control-group assignment resulted in contamination that negates reported treatment effectiveness. By applying the framework of the Rubin Causal Model (RCM) to isolate “pure” and “impure” experimental and control cases, the actual group assignment and not self-reported membership is shown to be a more accurate gauge of treatment level and effect. The analysis reveals that the form of contamination Rossi detected leads to underestimates of treatment effects, not their evaporation. While contamination is a legitimate threat in any research design its identification must be based on empirical measures. © 2003 by the Association for Public Policy Analysis and Management.