Evaluating the impact of policy research: Lessons from the evaluation of rural policy research in developing countries

Policy research, particularly when focused on the problems of developing countries, is under pressure by key investors to justify its impact compared to other types of development assistance that produce more tangible outcomes (e.g., roads, schools, and plant breeding). This pressure has led to a growing literature about appropriate methods for evaluating policy research as well as many evaluation case studies, particularly for policy research addressing agriculture, food and rural poverty issues rural policy research for short. This paper reviews the relevant literature and finds that while many studies have successfully demonstrated plausible narratives about how specific lines of policy research have contributed to policy change, few have gone the next step and quantified the impact of the policy change that was influenced, and even fewer have gone the final step to calculate a rate of economic or social return to the policy research investment being evaluated. This is in marked contrast to the rapid growth in rigorous quantitative evaluations of many other types of development assistance investments. This paper seeks to identify factors constraining the rigorous evaluation of more policy research, and draws on existing evaluation studies of rural policy research to identify best practice lessons for each of the steps involved in an impact evaluation. The paper concludes that although underlying conceptual, data, and measurement problems constrain what can be done for many types of policy research, there is still scope for more quantitative assessments of policy research. However, a first priority should be good qualitative evaluations of policy influence, both as a means to better understanding how policy research can be more influential, but also as a necessary first step in any evaluation for without demonstrated policy influence there is no need to evaluate policy outcomes and impacts. A cautionary note is that too tight a focus on quantitative evaluations by investors could inadvertently bias the policy research agenda towards problems that are amenable to rigorous assessment and not necessarily to those that are most important or beneficial.