IDEAS home Printed from https://ideas.repec.org/a/taf/jdevef/v4y2012i3p388-408.html
   My bibliography  Save this article

Systematic reviews: from ‘bare bones’ reviews to policy relevance

Author

Listed:
  • Birte Snilstveit

Abstract

Theory-based systematic reviews, which summarise evidence on what works, when and why, strive for greater policy relevance. Reviews that answer these questions adopt a mixed methods approach and draw on a range of study types. Answering the ‘what works’ and ‘what doesn't’ questions means drawing on effectiveness studies, conducted to standards of high-quality impact evaluation. But in formulating answers to the ‘when’ and ‘why’ questions requires a broader range of evidence from both quantitative and qualitative research. Based on a review of the methodological literature in this field and the experience of 3ie's systematic reviews program, this article provides an outline of how a theory-based approach to systematic reviews, including appropriate quantitative and qualitative evidence, can be operationalised. We propose an approach based on three principles: (1) develop a program theory for the intervention; (2) adopt a mixed methods approach and include a broader range of evidence; and (3) maintain the rigour and transparency that characterise systematic reviews. The approach translates into two broad options. Effectiveness plus reviews focus on providing a detailed causal chain analysis by drawing on a program theory and additional data collection on context and intervention implementation. And effectiveness plus with a parallel review component, which is designed to answer specific research questions related to effectiveness, adopts separate inclusion criteria, reflecting the type of studies appropriate for answering those questions.

Suggested Citation

  • Birte Snilstveit, 2012. "Systematic reviews: from ‘bare bones’ reviews to policy relevance," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 4(3), pages 388-408, September.
    • Handle: RePEc:taf:jdevef:v:4:y:2012:i:3:p:388-408
    DOI: 10.1080/19439342.2012.709875
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/19439342.2012.709875
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/19439342.2012.709875?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Howard White, 2009. "Theory-based impact evaluation: principles and practice," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 271-284.
    2. White, Howard, 2009. "Theory-Based Impact Evaluation," 3ie Publications 2009-3, International Initiative for Impact Evaluation (3ie).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Roy Carr‐Hill & Caine Rolleston & Tejendra Pherali & Rebecca Schendel, 2014. "PROTOCOL: The Effects of School‐Based Decision Making on Educational Outcomes in Low and Middle Income Contexts: A Systematic Review," Campbell Systematic Reviews, John Wiley & Sons, vol. 10(1), pages 1-46.
    2. Natalie Rebelo Da Silva & Hazel Zaranyika & Laurenz Langer & Nicola Randall & Evans Muchiri & Ruth Stewart, 2017. "Making the Most of What We Already Know," Evaluation Review, , vol. 41(2), pages 155-172, April.
    3. Sara Stevano & Suneetha Kadiyala & Deborah Johnston & Hazel Malapit & Elizabeth Hull & Sofia Kalamatianou, 2019. "Time-Use Analytics: An Improved Way of Understanding Gendered Agriculture-Nutrition Pathways," Feminist Economics, Taylor & Francis Journals, vol. 25(3), pages 1-22, July.
    4. Constanza Gonzalez Parrao & Marta Moratti & Shannon Shisler & Birte Snilstveit & John Eyers, 2021. "PROTOCOL: Aquaculture for improving productivity, income, nutrition and women's empowerment in low‐ and middle‐income countries: A systematic review and meta‐analysis," Campbell Systematic Reviews, John Wiley & Sons, vol. 17(3), September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kabeer, Naila, 2020. "‘Misbehaving’ RCTs: The confounding problem of human agency," World Development, Elsevier, vol. 127(C).
    2. Johnson, Nancy L. & Atherstone, Christine & Grace, Delia, 2015. "The potential of farm-level technologies and practices to contribute to reducing consumer exposure to aflatoxins: A theory of change analysis:," IFPRI discussion papers 1452, International Food Policy Research Institute (IFPRI).
    3. Cornwall, Andrea & Aghajanian, Alia, 2017. "How to Find out What’s Really Going On: Understanding Impact through Participatory Process Evaluation," World Development, Elsevier, vol. 99(C), pages 173-185.
    4. Naila Kabeer & Munshi Sulaiman, 2015. "Assessing the Impact of Social Mobilization: Nijera Kori and the Construction of Collective Capabilities in Rural Bangladesh," Journal of Human Development and Capabilities, Taylor & Francis Journals, vol. 16(1), pages 47-68, February.
    5. Sara Rafael Almeida & Joana Sousa Lourenco & Francois J. Dessart & Emanuele Ciriolo, 2017. "Insights from behavioural sciences to prevent and combat violence against women. Literature review," JRC Research Reports JRC103975, Joint Research Centre.
    6. Quentin Ssossé & Johanna Wagner & Carina Hopper, 2021. "Assessing the Impact of ESD: Methods, Challenges, Results," Sustainability, MDPI, vol. 13(5), pages 1-26, March.
    7. Joana Silva Afonso, 2020. "Impact evaluation, social performance assessment and standardisation: reflections from microfinance evaluations in Pakistan and Zimbabwe," Working Papers in Economics & Finance 2020-14, University of Portsmouth, Portsmouth Business School, Economics and Finance Subject Group.
    8. Oya, Carlos & Schaefer, Florian & Skalidou, Dafni, 2018. "The effectiveness of agricultural certification in developing countries: A systematic review," World Development, Elsevier, vol. 112(C), pages 282-312.
    9. Gwenolé Le Velly & Céline Dutilly, 2016. "Evaluating Payments for Environmental Services: Methodological Challenges," PLOS ONE, Public Library of Science, vol. 11(2), pages 1-20, February.
    10. Calina-Ana Butiu, 2017. "Evidence based practice in academic dropout policy. The pro-integra model," Journal of Community Positive Practices, Catalactica NGO, issue 1, pages 3-12.
    11. Channarith Meng & Wade Pfau, 2012. "Simulating the Impacts of Cash Transfers on Poverty and School Attendance: The Case of Cambodia," Journal of Family and Economic Issues, Springer, vol. 33(4), pages 436-452, December.
    12. Johnson, Nancy L. & Guedenet, Hannah & Saltzman, Amy, 2015. "What will it take for biofortification to have impact on the ground? Theories of change for three crop-country combinations:," IFPRI discussion papers 1427, International Food Policy Research Institute (IFPRI).
    13. Jorge García Hombrados & Maira Devisscher & María Herreros Martínez, 2015. "The Impact of land titling on agricultural production and agricultural investments in Tanzania: a theory-based approach," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 7(4), pages 530-544, December.
    14. Sarah Bracking, 2012. "How do Investors Value Environmental Harm/Care? Private Equity Funds, Development Finance Institutions and the Partial Financialization of Nature-based Industries," Development and Change, International Institute of Social Studies, vol. 43(1), pages 271-293, January.
    15. Henrik Hansen & Ole Winckler Andersen & Howard White, 2011. "Impact evaluation of infrastructure interventions," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 3(1), pages 1-8.
    16. Börner, Jan & Baylis, Kathy & Corbera, Esteve & Ezzine-de-Blas, Driss & Honey-Rosés, Jordi & Persson, U. Martin & Wunder, Sven, 2017. "The Effectiveness of Payments for Environmental Services," World Development, Elsevier, vol. 96(C), pages 359-374.
    17. Colin Kirkpatrick, 2012. "Economic Governance: Improving the Economic and Regulatory Environment for Supporting Private Sector Activity," WIDER Working Paper Series wp-2012-108, World Institute for Development Economic Research (UNU-WIDER).
    18. Jabeen, Sumera, 2016. "Do we really care about unintended outcomes? An analysis of evaluation theory and practice," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 144-154.
    19. Ron Bose, 2010. "CONSORT Extensions for Development Effectiveness: guidelines for the reporting of randomised control trials of social and economic policy interventions in developing countries," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 2(1), pages 173-186.
    20. Pablo Vidueira & José M. Díaz-Puente & María Rivera, 2014. "Socioeconomic Impact Assessment in Ex Ante Evaluations," Evaluation Review, , vol. 38(4), pages 309-335, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jdevef:v:4:y:2012:i:3:p:388-408. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/RJDE20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.