IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v34y2011i2p135-146.html
   My bibliography  Save this article

Ten steps to making evaluation matter

Author

Listed:
  • Sridharan, Sanjeev
  • Nakaima, April

Abstract

This paper proposes ten steps to make evaluations matter. The ten steps are a combination of the usual recommended practice such as developing program theory and implementing rigorous evaluation designs with a stronger focus on more unconventional steps including developing learning frameworks, exploring pathways of evaluation influence, and assessing spread and sustainability. Consideration of these steps can lead to a focused dialogue between program planners and evaluators and can result in more rigorously planned programs. The ten steps can also help in developing and implementing evaluation designs that have greater potential for policy and programmatic influence. The paper argues that there is a need to go beyond a formulaic approach to program evaluation design that often does not address the complexity of the programs. The complexity of the program will need to inform the design of the evaluation. The ten steps that are described in this paper are heavily informed by a Realist approach to evaluation. The Realist approach attempts to understand what is it about a program that makes it work.

Suggested Citation

  • Sridharan, Sanjeev & Nakaima, April, 2011. "Ten steps to making evaluation matter," Evaluation and Program Planning, Elsevier, vol. 34(2), pages 135-146, May.
    • Handle: RePEc:eee:epplan:v:34:y:2011:i:2:p:135-146
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149-7189(10)00081-9
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Pluye, Pierre & Potvin, Louise & Denis, Jean-Louis, 2004. "Making public health programs last: conceptualizing sustainability," Evaluation and Program Planning, Elsevier, vol. 27(2), pages 121-133, May.
    2. Paul R. Carlile, 2004. "Transferring, Translating, and Transforming: An Integrative Framework for Managing Knowledge Across Boundaries," Organization Science, INFORMS, vol. 15(5), pages 555-568, October.
    3. Michael Woolcock, 2009. "Toward a plurality of methods in project evaluation: a contextualised approach to understanding impact trajectories and efficacy," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(1), pages 1-14.
    4. Pluye, Pierre & Potvin, Louise & Denis, Jean-Louis, 2004. "Corrigendum to `Making public health programs last: conceptualizing sustainability' [Evaluation and Program Planning 27 (2004) 121-133]," Evaluation and Program Planning, Elsevier, vol. 27(4), pages 453-453, November.
    5. Johnson, Knowlton & Hays, Carol & Center, Hayden & Daley, Charlotte, 2004. "Building capacity and sustainable prevention innovations: a sustainability planning model," Evaluation and Program Planning, Elsevier, vol. 27(2), pages 135-149, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sridharan, Sanjeev & Jones, Bobby & Caudill, Barry & Nakaima, April, 2016. "Steps towards incorporating heterogeneities into program theory: A case study of a data-driven approach," Evaluation and Program Planning, Elsevier, vol. 58(C), pages 88-97.
    2. Sridharan, Sanjeev & Nakaima, April, 2023. "Learning from experiences of evaluators implementing theory-driven evaluations in diverse settings: Building on the contributions of John Mayne," Evaluation and Program Planning, Elsevier, vol. 97(C).
    3. Cole, Donald C. & Levin, Carol & Loechl, Cornelia & Thiele, Graham & Grant, Frederick & Girard, Aimee Webb & Sindi, Kirimi & Low, Jan, 2016. "Planning an integrated agriculture and health program and designing its evaluation: Experience from Western Kenya," Evaluation and Program Planning, Elsevier, vol. 56(C), pages 11-22.
    4. Clark, Alexander M., 2013. "What are the components of complex interventions in healthcare? Theorizing approaches to parts, powers and the whole intervention," Social Science & Medicine, Elsevier, vol. 93(C), pages 185-193.
    5. Arasanz, Carla & Nylen, Kirk, 2020. "The theory of change of the evaluation support program: Enhancing the role of community organizations in providing an ecology of care for neurological disorders," Evaluation and Program Planning, Elsevier, vol. 80(C).
    6. Schultes, Marie-Therese & Kollmayer, Marlene & Mejeh, Mathias & Spiel, Christiane, 2018. "Attitudes toward evaluation: An exploratory study of students’ and stakeholders’ social representations," Evaluation and Program Planning, Elsevier, vol. 70(C), pages 44-50.
    7. Juan, Parra, 2017. "¿Qué Funciona, Para Quién, En Qué Aspectos, Hasta Qué Punto, En Qué Contexto Y Cómo? Una Introducción A La Evaluación Realista Y Sus Métodos [What Works, For Whom, In What Respects, To What Extent,," MPRA Paper 81137, University Library of Munich, Germany.
    8. Juan David Parra, 2017. "Introducción a la evaluación realista y sus métodos: ¿Qué funciona, para quién, en qué aspectos, hasta qué punto, en qué contexto y cómo?," Revista Economía y Región, Universidad Tecnológica de Bolívar, vol. 11(2), pages 11-44, December.
    9. Nordhagen, Stella & Nielsen, Jennifer & van Mourik, Tom & Smith, Erin & Klemm, Rolf, 2019. "Fostering CHANGE: Lessons from implementing a multi-country, multi-sector nutrition-sensitive agriculture project," Evaluation and Program Planning, Elsevier, vol. 77(C).
    10. Okwaro, Ferdinand M. & Chandler, Clare I.R. & Hutchinson, Eleanor & Nabirye, Christine & Taaka, Lilian & Kayendeke, Miriam & Nayiga, Susan & Staedke, Sarah G., 2015. "Challenging logics of complex intervention trials: Community perspectives of a health care improvement intervention in rural Uganda," Social Science & Medicine, Elsevier, vol. 131(C), pages 10-17.
    11. Tannahill, Carol & Sridharan, Sanjeev, 2013. "Getting real about policy and practice needs: Evaluation as a bridge between the problem and solution space," Evaluation and Program Planning, Elsevier, vol. 36(1), pages 157-164.
    12. Mehdipanah, Roshanak & Manzano, Ana & Borrell, Carme & Malmusi, Davide & Rodriguez-Sanz, Maica & Greenhalgh, Joanne & Muntaner, Carles & Pawson, Ray, 2015. "Exploring complex causal pathways between urban renewal, health and health inequality using a theory-driven realist approach," Social Science & Medicine, Elsevier, vol. 124(C), pages 266-274.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alfonso, Moya L. & Nickelson, Jen & Hogeboom, David L. & French, Jennifer & Bryant, Carol A. & McDermott, Robert J. & Baldwin, Julie A., 2008. "Assessing local capacity for health intervention," Evaluation and Program Planning, Elsevier, vol. 31(2), pages 145-159, May.
    2. Sridharan, Sanjeev & Go, Sodam & Zinzow, Heidi & Gray, Aracelis & Gutierrez Barrett, Melissa, 2007. "Analysis of strategic plans to assess planning for sustainability of comprehensive community initiatives," Evaluation and Program Planning, Elsevier, vol. 30(1), pages 105-113, February.
    3. Johnson, K.W. & Grube, J.W. & Ogilvie, K.A. & Collins, D. & Courser, M. & Dirks, L.G. & Ogilvie, D. & Driscoll, D., 2012. "A community prevention model to prevent children from inhaling and ingesting harmful legal products," Evaluation and Program Planning, Elsevier, vol. 35(1), pages 113-123.
    4. Scheirer, Mary Ann & Hartling, Gretchen & Hagerman, Diane, 2008. "Defining sustainability outcomes of health programs: Illustrations from an on-line survey," Evaluation and Program Planning, Elsevier, vol. 31(4), pages 335-346, November.
    5. Williams, Randi M. & Zhang, Jing & Woodard, Nathaniel & Slade, Jimmie & Santos, Sherie Lou Zara & Knott, Cheryl L., 2020. "Development and validation of an instrument to assess institutionalization of health promotion in faith-based organizations," Evaluation and Program Planning, Elsevier, vol. 79(C).
    6. Douglas S. Gardner & Jacob M. Marszalek, 2014. "Does the Sustainability of Comprehensive Intervention Programs Depend on Parent Involvement?," SAGE Open, , vol. 4(3), pages 21582440145, August.
    7. Toledo Romani, Maria E. & Vanlerberghe, Veerle & Perez, Dennis & Lefevre, Pierre & Ceballos, Enrique & Bandera, Digna & Baly Gil, Alberto & Van der Stuyft, Patrick, 2007. "Achieving sustainability of community-based dengue control in Santiago de Cuba," Social Science & Medicine, Elsevier, vol. 64(4), pages 976-988, February.
    8. Riki Savaya & Gerald Elsworth & Patricia Rogers, 2009. "Projected Sustainability of Innovative Social Programs," Evaluation Review, , vol. 33(2), pages 189-205, April.
    9. Sebastian Ion Ceptureanu & Eduard Gabriel Ceptureanu & Cristian Eugen Luchian & Iuliana Luchian, 2018. "Community Based Programs Sustainability. A Multidimensional Analysis of Sustainability Factors," Sustainability, MDPI, vol. 10(3), pages 1-15, March.
    10. Sachiko Ozawa & Suneeta Singh & Kriti Singh & Vibha Chhabra & Sara Bennett, 2016. "The Avahan Transition: Effects of Transition Readiness on Program Institutionalization and Sustained Outcomes," PLOS ONE, Public Library of Science, vol. 11(7), pages 1-15, July.
    11. Baoan Song & Qiyu Sun & Ying Li & Chuanqi Que, 2016. "Evaluating the Sustainability of Community-Based Long-Term Care Programmes: A Hybrid Multi-Criteria Decision Making Approach," Sustainability, MDPI, vol. 8(7), pages 1-19, July.
    12. Lawrence A Palinkas & Claudia V Chavarin & Claudia M Rafful & Mee Young Um & Doroteo V Mendoza & Hugo Staines & Gregory A Aarons & Thomas L Patterson, 2015. "Sustainability of Evidence-Based Practices for HIV Prevention among Female Sex Workers in Mexico," PLOS ONE, Public Library of Science, vol. 10(10), pages 1-17, October.
    13. Najafizada, Said Ahmad Maisam & Labonté, Ronald & Bourgeault, Ivy Lynn, 2017. "Stakeholder’s perspective: Sustainability of a community health worker program in Afghanistan," Evaluation and Program Planning, Elsevier, vol. 60(C), pages 123-129.
    14. Sebastian Ion Ceptureanu & Eduard Gabriel Ceptureanu, 2019. "Community-Based Healthcare Programs Sustainability Impact on the Sustainability of Host Organizations: A Structural Equation Modeling Analysis," IJERPH, MDPI, vol. 16(20), pages 1-18, October.
    15. Bennett, Sara & Ozawa, Sachiko & Rodriguez, Daniela & Paul, Amy & Singh, Kriti & Singh, Suneeta, 2015. "Monitoring and evaluating transition and sustainability of donor-funded programs: Reflections on the Avahan experience," Evaluation and Program Planning, Elsevier, vol. 52(C), pages 148-158.
    16. Anne Casati & Corine Genet, 2014. "Principal investigators as scientific entrepreneurs," The Journal of Technology Transfer, Springer, vol. 39(1), pages 11-32, February.
    17. Jessica Persson & Ann Svensson & Ingela Grönbeck Lindén & Sven Kylén & Catharina Hägglin, 2022. "Aspects of Expansive Learning in the Context of Healthy Ageing—A Formative Intervention between Dental Care and Municipal Healthcare," IJERPH, MDPI, vol. 19(3), pages 1-16, January.
    18. Emmanuelle Vaast & Geoff Walsham, 2009. "Trans-Situated Learning: Supporting a Network of Practice with an Information Infrastructure," Information Systems Research, INFORMS, vol. 20(4), pages 547-564, December.
    19. Daniele T. P. Souza & Eugenia A. Kuhn & Arjen E. J. Wals & Pedro R. Jacobi, 2020. "Learning in, with, and through the Territory: Territory-Based Learning as a Catalyst for Urban Sustainability," Sustainability, MDPI, vol. 12(7), pages 1-19, April.
    20. Marte C.W. Solheim & Ron Boschma & Sverre Herstad, 2018. "Related variety, unrelated variety and the novelty content of firm innovation in urban and non-urban locations," Papers in Evolutionary Economic Geography (PEEG) 1836, Utrecht University, Department of Human Geography and Spatial Planning, Group Economic Geography, revised Oct 2018.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:34:y:2011:i:2:p:135-146. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.