IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v91y2022ics0149718922000106.html

What does it mean for an evaluation to be ‘valid’? A critical synthesis of evaluation literature

Author

Listed:
  • Downes, Jenni
  • Gullickson, Amy M.

Abstract

Validity is both a critical foundation for, and widely contested concept in the field of program evaluation. Given the wide and divergent treatment of the term, this article sought to enhance the field’s understanding and application of validity by systematically identifying, collating and integrating existing conceptions of validity. A critical interpretive synthesis approach was taken to combine the best elements of systematic literature searching with integrative review. We found that conceptions of validity could be roughly collated into three categories pertaining to the evaluation research, evaluative judgement and reporting, and beyond the evaluation proper. The result was a wide-ranging map of validity concepts against the various stages of the evaluation process. This made clear several gaps in the validity thinking, resulting in the development of comprehensive but not yet complete framework that weaves together related aspects identified into a cohesive whole. It is our hope that this initial iteration of a cohesive validity framework will be the starting point for a cooperative effort to improve professional evaluation practice.

Suggested Citation

  • Downes, Jenni & Gullickson, Amy M., 2022. "What does it mean for an evaluation to be ‘valid’? A critical synthesis of evaluation literature," Evaluation and Program Planning, Elsevier, vol. 91(C).
    • Handle: RePEc:eee:epplan:v:91:y:2022:i:c:s0149718922000106
    DOI: 10.1016/j.evalprogplan.2022.102056
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718922000106
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2022.102056?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Patton, Michael Quinn, 1989. "A context and boundaries for a theory-driven approach to validity," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 375-377, January.
    2. Palumbo, Dennis J. & Oliverio, Annamarie, 1989. "Implementation theory and the theory-driven approach to validity," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 337-344, January.
    3. Chen, Huey-Tsyh & Rossi, Peter H., 1987. "The theory-driven approach to validity," Evaluation and Program Planning, Elsevier, vol. 10(1), pages 95-103, January.
    4. Arbour, Ghislain, 2020. "Teaching programme evaluation: A problem of knowledge," Evaluation and Program Planning, Elsevier, vol. 83(C).
    5. Chen, Huey T., 2010. "The bottom-up approach to integrative validity: A new perspective for program evaluation," Evaluation and Program Planning, Elsevier, vol. 33(3), pages 205-214, August.
    6. Garaway, G. B., 1997. "Evaluation, validity, and values," Evaluation and Program Planning, Elsevier, vol. 20(1), pages 1-5, February.
    7. Gullickson, Amy M. & King, Jean A. & LaVelle, John M. & Clinton, Janet M., 2019. "The current state of evaluator education: A situation analysis and call to action," Evaluation and Program Planning, Elsevier, vol. 75(C), pages 20-30.
    8. Benjamin Saunders & Julius Sim & Tom Kingstone & Shula Baker & Jackie Waterfield & Bernadette Bartlam & Heather Burroughs & Clare Jinks, 2018. "Saturation in qualitative research: exploring its conceptualization and operationalization," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1893-1907, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ogundeji, Yewande Kofoworola & Quinn, Amity & Lunney, Meaghan & Chong, Christy & Chew, Derek & Danso, George & Duggan, Shelly & Edwards, Alun & Hopkin, Gareth & Senior, Peter & Sumner, Glen & Williams, 2021. "Factors that influence specialist physician preferences for fee-for-service and salary-based payment models: A qualitative study," Health Policy, Elsevier, vol. 125(4), pages 442-449.
    2. Jennifer Tartaglia & Michelle McIntosh & Jonine Jancey & Jane Scott & Andrea Begley, 2021. "Exploring Feeding Practices and Food Literacy in Parents with Young Children from Disadvantaged Areas," IJERPH, MDPI, vol. 18(4), pages 1-18, February.
    3. Lulu Lunogelo Sakafu & Godfrey Sama Philipo & Christina V Malichewe & Lulu S Fundikira & Flora A Lwakatare & Katherine Van Loon & Beatrice P Mushi & Rebecca J DeBoer & Stella A Bialous & Amie Y Lee, 2022. "Delayed diagnostic evaluation of symptomatic breast cancer in sub-Saharan Africa: A qualitative study of Tanzanian women," PLOS ONE, Public Library of Science, vol. 17(10), pages 1-15, October.
    4. Ingrid van Putten & Catherine Longo & Ashleigh Arton & Matt Watson & Christopher M Anderson & Amber Himes-Cornell & Clara Obregón & Lucy Robinson & Tatiana van Steveninck, 2020. "Shifting focus: The impacts of sustainable seafood certification," PLOS ONE, Public Library of Science, vol. 15(5), pages 1-24, May.
    5. Najwa Taghy & Linda Cambon & Caroline Boulliat & Olivier Aromatario & Claude Dussart, 2021. "Exploring the Determinants of Polypharmacy Prescribing and Dispensing Behaviours in Primary Care for the Elderly—Protocol for a Qualitative Study," IJERPH, MDPI, vol. 18(14), pages 1-10, July.
    6. Sara M. London & Jessica D. Hanson & Michelle Sarche & Kyra Oziel & Dedra Buchwald, 2024. "Reach, Acceptability, and Sustainability of the Native Changing High-Risk Alcohol Use and Increasing Contraception Effectiveness Study (CHOICES) Intervention: A Qualitative Evaluation of an Alcohol-Exposed Pregnancy Prevention Program," IJERPH, MDPI, vol. 21(3), pages 1-10, February.
    7. Christopher Holmberg & Eric Carlström & Helena Collier, 2020. "Registered nurses' perspectives on medically safe practices and sound ethical standards in aesthetic nursing: An interview study," Journal of Clinical Nursing, John Wiley & Sons, vol. 29(5-6), pages 944-954, March.
    8. Kwasniok, Sascha, 2025. "Planned behavior, insurance knowledge and the demand for private disability insurance – Evidence from Germany," EconStor Preprints 321819, ZBW - Leibniz Information Centre for Economics.
    9. Vandana Madhavan & Murale Venugopalan & Bhumika Gupta & Gyanendra Singh Sisodia, 2023. "Addressing Agency Problem in Employee Training: The Role of Goal Congruence," Sustainability, MDPI, vol. 15(4), pages 1-27, February.
    10. Cerezo, M. Angeles & Dasi, Carmen & Ruiz, Juan Carlos, 2013. "Supporting parenting of infants: Evaluating outcomes for parents and children in a community-based program," Evaluation and Program Planning, Elsevier, vol. 37(C), pages 12-20.
    11. Khezar Hayat & Meagen Rosenthal & Ali Hassan Gillani & Panpan Zhai & Muhammad Majid Aziz & Wenjing Ji & Jie Chang & Hao Hu & Yu Fang, 2019. "Perspective of Pakistani Physicians towards Hospital Antimicrobial Stewardship Programs: A Multisite Exploratory Qualitative Study," IJERPH, MDPI, vol. 16(9), pages 1-15, May.
    12. Nixon Lemlem & Prof. Patrick Karanja Ngugi & Dr. Samson Nyang’au Paul, 2025. "Project Stakeholders’ Participation and Implementation of Water Projects in Kenya," Journal of Entrepreneurship and Project Management, CARI Journals Limited, vol. 10(3), pages 66-82.
    13. Van Droogenbroeck, Ellen & Van Hove, Leo, 2020. "Intra-household task allocation in online grocery shopping: Together alone," Journal of Retailing and Consumer Services, Elsevier, vol. 56(C).
    14. Thuy Séran & Anne-Sophie Fernandez & Hervé Chappert, 2024. "Managing coopetition in multi-unit organizations: a management-control perspective," Review of Managerial Science, Springer, vol. 18(10), pages 2889-2924, October.
    15. Cath, Corinne, 2021. "The technology we choose to create: Human rights advocacy in the Internet Engineering Task Force," Telecommunications Policy, Elsevier, vol. 45(6).
    16. Luswata Rogers, 2025. "A Strategy to Improve the Agricultural Loan Portfolio in Finance Trust Bank," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 9(9), pages 3838-3874, September.
    17. Noëllie Dunand & Philippe Golay & Charles Bonsack & Danièle Spagnoli & Valentino Pomini, 2024. "Good psychiatric management for borderline personality disorder: A qualitative study of its implementation in a supported employment team," PLOS ONE, Public Library of Science, vol. 19(3), pages 1-14, March.
    18. Karina Soubra & Camilla Tamworth & Zeast Kamal & Clare Brook & Dawn Langdon & Jo Billings, 2023. "Health and social care workers experiences of coping while working in the frontline during the COVID-19 pandemic: One year on," PLOS ONE, Public Library of Science, vol. 18(4), pages 1-27, April.
    19. Carla Dias-Barbosa & Natalie Taylor & Kevin Chan & Jean Philippe York & Jorge Puelles & Rajeev Chavda, 2026. "Patient Insights on Acne Hyperpigmentation, Trifarotene, and Skincare: 6-Month, LEAP Phase 4 Qualitative Exit Interview Findings," The Patient: Patient-Centered Outcomes Research, Springer;International Academy of Health Preference Research, vol. 19(2), pages 343-352, March.
    20. Şahika Simsek-Cetinkaya & Simge Evrenol Ocal, 2023. "“Psychological Injuries Are Not Visible†: Experiences and Perceptions of Midwives and Nurses about Domestic Violence during Pregnancy," Clinical Nursing Research, , vol. 32(8), pages 1115-1123, November.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:91:y:2022:i:c:s0149718922000106. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.