Aftershocks of Chile’s Earthquake for an Ongoing, Large-Scale Experimental Evaluation

Evaluation designs for social programs are developed assuming minimal or no disruption from external shocks, such as natural disasters. This is because extremely rare shocks may not make it worthwhile to account for them in the design. Among extreme shocks is the 2010 Chile earthquake. Un Buen Comienzo (UBC), an ongoing early childhood program in Chile, was directly affected by the earthquake. This article discusses (a) the factors the UBC team considered for deciding whether to put on hold or continue implementation and data collection for this experimental study; and (b) how the team reached consensus on those decisions. A lesson learned is that the use of an experimental design for UBC insured that the evaluation’s internal validity was not compromised by the earthquake’s consequences, although cohort comparisons were compromised. Other lessons can be transferred to other contexts where external shocks affect an ongoing experimental or quasi-experimental impact evaluation.