Discovering treatment effect heterogeneity through post‐treatment variables with application to the effect of class size on mathematics scores

Class size reduction represents an appealing but costly education policy lever. A crucial question, therefore, is whether certain types of students benefit particularly from small classes, i.e. are there effect modifiers for the benefit of small classes? We use data from the Tennessee ‘Student/teacher achievement ratio’ study, which is a large randomized experiment of class size to address this question. In the Tennessee study, a challenge is that relatively few potential effect modifiers were measured at baseline. Yet many proxies for effect modifiers were measured after baseline. In an experiment, treatment effect modification based on pretreatment variables can be assessed by using standard regression techniques. For post‐treatment variables, however, such regression approaches are valid only under a strong sequential ignorability assumption. We develop two instrumental‐variable‐based approaches for studying effect modification based on post‐treatment variables. The key advantage of the method proposed is that it allows relaxation of the strong sequential ignorability assumption. However, this advantage comes at the cost of reliance on alternative assumptions that can be challenging to verify. We provide evidence that students who are not academically engaged outside the classroom benefit more from small classes than students who are academically engaged.