Causal Inference and the Millennium Development Goals (MDGs): Assessing Whether There Was an Acceleration in MDG Development Indicators Following the MDG Declaration

Background: The Millennium Development Goals (MDGs) are a set of eight goals and corresponding indicators that were agreed to following the adoption of the United Nations Millennium Declaration in September 2000 by leaders of 189 countries. The goals state specific objectives for the world to accomplish by measuring progress in indicators during the time period from 1990 (ten years before the declaration) to 2015. While monitoring mechanisms have reported the progress towards achieving these goals, there has been little effort to evaluate whether there was a change in the development outcomes associated with the activities initiated by the MDGs. The dearth of evaluations applied to the MDGs may be associated with the lack of a true counterfactual or the challenges with the data quality. Our analysis focused on the questions of whether there was a statistically significant acceleration or deceleration (mathematically defined as an interrupted slope or intercept) for a particular indicator and, if there was one, whether that changepoint occurred before or after 2001. Accelerations occurring in 2000 or earlier cannot be causally associated with the MDG-related activities (since the acceleration predated the declaration) while accelerations after 2000 may logically be associated with MDG-related activities. Method: We applied the standard program evaluation methodology of an interrupted time series to the country level yearly measurements of the MDG indicators as well as a set of control indicators that were not included in the set of MDG indicators (and were not likely to have been directly impacted by MDG-related activities). The modeling technique used was a multiple linear mixed model where we identified the optimal year of the changepoint in the outcome by examining years 1992 to 2008 for all datasets. Analysis was performed separately for IDA-only countries (World Bank 2000 designation) as well as for a broader set of countries consisting of IDA, IBRD and Blend countries. The IDA (International Development Association) focuses on low income countries and the IBRD (International Bank for Reconstruction and Development) focuses on middle income countries. The primary data source for the analysis was the World Bank database where the analysis explicitly assumes that the reported data points are accurate. Reported results contain separate analyses for (1) including heavy influence countries and (2) excluding heavy influence countries; thus resulting in four sets of reported analyses as well as a detailed review of the individual MDG indicator. Results: The general result was that there was no trend in statistically significant accelerations in the MDG indicators after 2000. Rather the results for all four sets of reported analysis were consistent in that about half of the MDG indicators exhibited no acceleration or deceleration during the time period from 1992 to 2008 and about one-third exhibited accelerations BEFORE 2001. Contrarily, nearly all of the control indicators had no change (neither acceleration nor deceleration) during the time period. It should be emphasized that the control indicators were identified based on data availability and other control indicators may exist that serve as more appropriate controls. The only MDG indicator that had a statistically significant acceleration in progress in 2001 or after for all four datasets was Indicator 8D (Debt Service). MDG indicators 1B (GDP per person employed), 4A (Infant Mortality Rate), 4A (Under-five mortality rate), and 6A (HIV prevalence among population aged 15-24 years) all had statistically significant accelerations in 2000 or earlier for all four sets of analysis. Discussion: The results may reflect some of the historical nature of the MGDs in that the Millennium Declaration represented a culmination of development agreements and goals that had been established over the preceding years. As such, many of the indicators selected to belong to the MDGs in 2000 had been previously identified in the global development agenda in the 1990s and campaigns to accelerate progress had been initiated before 2000. In fact, when the results of this study have been demonstrated at different United Nations forums, the reaction from seasoned development professionals has consistently been that of affirmation, where the audience generally has indicated that intuitively they would have expected the observed results given their knowledge of how the MDG indicators had been identified. Additionally, the results may be indicative of the impact of long-term broader economic trends where, for example, official development assistance (ODA) comprises only a very minor part of the global economy. Conclusion: In order to ensure ongoing global support for development, especially as the global development community looks beyond 2015 to the post-MDG era, there needs to be careful communication regarding what the MDGs did and did not accomplish. Many of those who are not development professionals have been drawn into the global discussions due to the strong communications support for the Millennium Development Goals. As such, much of the general public expects that an acceleration in progress was triggered following the September 2000 Declaration and the associated increase in ODA. Contrarily, the data show clearly that the activities following the MDG Declaration did not provide an acceleration in most of the development goals. For the subset of MDG indicators that experienced an acceleration, the accelerations tended to occur before the MDG Declaration. This does not preclude the possibility that activities associated with the MDGs helped sustain accelerations that predated the September 2000 MDG Declaration or ward off decelerations. Additionally, this does not preclude the possibility that the MDGS provided an acceleration post-2000 in a subset of countries (i.e. sub-Saharan Africa or heavily ODA dependent countries) but it is important to note that this study examined the broadest question regarding the impact of the MDGs on the entire set of developing countries rather than a selected subset.