Multiple Testing And Heterogeneous Treatment Effects: Re‐Evaluating The Effect Of Progresa On School Enrollment

SUMMARY The effect of a program or treatment may vary according to observed characteristics. In such a setting, it may not only be of interest to determine whether the program or treatment has an effect on some sub‐population defined by these observed characteristics, but also to determine for which sub‐populations, if any, there is an effect. This paper treats this problem as a multiple testing problem in which each null hypothesis in the family of null hypotheses specifies whether the program has an effect on the outcome of interest for a particular sub‐population. We develop our methodology in the context of PROGRESA, a large‐scale poverty‐reduction program in Mexico. In our application, the outcome of interest is the school enrollment rate and the sub‐populations are defined by gender and highest grade completed. Under weak assumptions, the testing procedure we construct controls the familywise error rate—the probability of even one false rejection—in finite samples. Similar to earlier studies, we find that the program has a significant effect on the school enrollment rate, but only for a much smaller number of sub‐populations when compared to results that do not adjust for multiple testing. Copyright © 2013 John Wiley & Sons, Ltd.