Targeting Policies: Multiple Testing and Distributional Treatment Effects

Economic theory often predicts that treatment responses may depend on individuals’ characteristics and location on the outcome distribution. Policymakers need to account for such treatment effect heterogeneity in order to efficiently allocate resources to subgroups that can successfully be targeted by a policy. However, when interpreting treatment effects across subgroups and the outcome distribution, inference has to be adjusted for multiple hypothesis testing to avoid an overestimation of positive treatment effects. We propose six new tests for treatment effect heterogeneity that make corrections for the family-wise error rate and that identify subgroups and ranges of the outcome distribution exhibiting economically and statistically significant treatment effects. We apply these tests to individual responses to welfare reform and show that welfare recipients benefit from the reform in a smaller range of the earnings distribution than previously estimated. Our results shed new light on effectiveness of welfare reform and demonstrate the importance of correcting for multiple testing.