Estimating Treatment-Effect Heterogeneity across Sites, in Multi-Site Randomized Experiments with Few Units per Site

Multi-site randomized experiments, which are common in economics and medicine, can reveal how treatment effects vary across settings. However, differences between site-specific estimates reflect both treatment-effect heterogeneity and estimation noise. We build on established measurement-error corrections and develop new ones to study this heterogeneity in experiments with many sites and few units per site. In particular, our results can be used to estimate the variance and centered third moment of treatment effects across sites, relate site effects to untreated outcomes, conduct mediation analyses, assess the roles of provider effectiveness and site responsiveness, and study heterogeneity in local average treatment effects (LATEs) under imperfect compliance. We apply our methods to counseling programs delivered by public and private providers in about 200 French employment offices. Average employment gains of about 2 percentage points conceal cross-site standard deviations of intention-to-treat effects of 8 to 9 percentage points. We also find substantial LATE heterogeneity. Effects are negatively associated with untreated employment rates. The two programs also tend to be more effective in the same sites, consistent with a role for heterogeneous site responsiveness.