Matching the gold standard: Comparing experimental and non-experimental evaluation techniques for a geographically targeted program

We compare non-experimental impact estimates based on matching methods with those from a randomized evaluation to determine whether the non-experimental approach can “match” the so-called gold standard. The social experiment we use was carried out to evaluate a geographically targeted conditional cash transfer antipoverty program in Nicaragua. The outcomes we assess include several components of household expenditure and a variety of children’s health outcomes including breast feeding, vaccinations, and morbidity. We find that using each of the following improves performance of matching for these outcomes: 1) geographically proximate comparison samples; 2) stringent common support requirements; and 3) both geographic- and household-level matching variables. Even for a geographically targeted program, in which the selection is at the geographic-, rather than at the individual- or household-level, and in which it is not possible to find comparison individuals or households in the program locales, matching can perform reasonably well. The results also suggest that the techniques may be more promising for evaluating the more easily measured individual-level binary outcomes, than for outcomes that are more difficult to measure, such as expenditure.