Have US-Funded CARSI Programs Reduced Crime and Violence in Central America? An Examination of LAPOP’S Impact Assessment of US Violence Prevention Programs in Central America

In October 2014, the Latin American Public Opinion Project (LAPOP) at Vanderbilt University published an impact assessment study of community-based violence prevention programs that have been implemented under the umbrella of the US State Department’s Central American Regional Security Initiative (CARSI). The study looked at survey data measuring public perceptions of crime in 127 treatment and control neighborhoods in municipalities in El Salvador, Guatemala, Honduras, and Panama where the violence prevention programs have been implemented. The study’s authors stated that the data shows that “in several key respects the programs have been a success” and note, for instance, that 51 percent fewer residents of “treated” communities reported being aware of murders and extortion incidents during the previous 12 months, and 19 percent fewer residents reported having heard about robberies having occurred. As the LAPOP study is, to date, the only publicly accessible impact assessment of programs carried out under CARSI — a notoriously opaque regional assistance scheme that has received hundreds of millions of dollars of US government funding — a thorough review of the LAPOP study data seemed appropriate. The following report examines the data collected during the LAPOP study and subjects them to a number of statistical tests. The authors find that the study cannot support the conclusion that the areas subject to treatment in the CARSI programs showed better results than those areas that were not. This report identifies major problems with the LAPOP study, namely, the nonrandomness of the selection of treatment versus control areas and how the differences in initial conditions, as well as differences in results between treatment and control areas, have been interpreted. In the case of reported robberies, if the areas subject to treatment have an elevated level of reported robberies in the year prior to treatment, it is possible that there is some reversion to normal levels over the next year. The LAPOP methodology does not differentiate between effective treatment and, for example, an unrelated decline in reported robberies in a treated area following a year with an abnormally high number of reported robberies. The series of statistical tests in this report indicate that this possibility is quite plausible, and cannot be ruled out; and that the LAPOP study, therefore, does not demonstrate a statistically significant positive effect of treatment. The same can be said for the other variables where the LAPOP study finds significant improvement.