Cost-Effectiveness Analysis in Development: Accounting for Local Costs and Noisy Impacts

Effective evidence-based policy making in development requires rigorous measurement of costs as well as impacts. This paper discusses important challenges and relevant solutions for implementing cost-effectiveness analysis and comparing the relative cost-effectiveness of programs across settings in the context of education. Adapting development programs from one context to another requires many assumptions. Most of the discussion of those assumptions, to date, has focused on the context-specificity (or external validity) of impact estimates. This study examines the sensitivity of cost-effectiveness analysis to errors in impact estimates, as well as the sensitivity of costs to context, and explores how biases such as recall and pilot bias may lead to over- or under-estimates of cost-effectiveness. We use data on the cost-effectiveness of 27 student learning programs and 16 attendance-boosting programs across Africa and Asia to demonstrate the magnitude of these challenges, as well as potential solutions. We show that comparing Monte Carlo simulations of cost-effectiveness to certain benchmarks and adapting the largest cost elements to local prices (i.e., parameter variation analysis) can remove much of the uncertainty surrounding cost-effectiveness estimates, and we propose that biases can be minimized through the use of detailed templates for cost reporting at the time of program implementation. Thus, this paper not only provides evidence of previously neglected challenges for cost-effectiveness analysis, but – more importantly – provides practical recommendations for undertaking this crucial stage in development planning well.