IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v45y2014icp119-126.html
   My bibliography  Save this article

Applying complexity theory: A review to inform evaluation design

Author

Listed:
  • Walton, Mat

Abstract

Complexity theory has increasingly been discussed and applied within evaluation literature over the past decade. This article reviews the discussion and use of complexity theory within academic journal literature. The aim is to identify the issues to be considered when applying complexity theory to evaluation. Reviewing 46 articles, two groups of themes are identified. The first group considers implications of applying complexity theory concepts for defining evaluation purpose, scope and units of analysis. The second group of themes consider methodology and method. Results provide a starting point for a configuration of an evaluation approach consistent with complexity theory, whilst also identifying a number of design considerations to be resolved within evaluation planning.

Suggested Citation

  • Walton, Mat, 2014. "Applying complexity theory: A review to inform evaluation design," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 119-126.
    • Handle: RePEc:eee:epplan:v:45:y:2014:i:c:p:119-126
    DOI: 10.1016/j.evalprogplan.2014.04.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S014971891400041X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2014.04.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Blackman, Tim & Wistow, Jonathan & Byrne, David, 2011. "A Qualitative Comparative Analysis of factors associated with trends in narrowing health inequalities in England," Social Science & Medicine, Elsevier, vol. 72(12), pages 1965-1974, June.
    2. Ssengooba, Freddie & McPake, Barbara & Palmer, Natasha, 2012. "Why performance-based contracting failed in Uganda – An “open-box” evaluation of a complex health system intervention," Social Science & Medicine, Elsevier, vol. 75(2), pages 377-383.
    3. -, 1998. "Evaluación de Quito," Copublicaciones, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL), number 2058.
    4. Graham Room, 2011. "Complexity, Institutions and Public Policy," Books, Edward Elgar Publishing, number 14394.
    5. Cabrera, Derek & Colosi, Laura & Lobdell, Claire, 2008. "Systems thinking," Evaluation and Program Planning, Elsevier, vol. 31(3), pages 299-310, August.
    6. Munda, Giuseppe, 2004. "Social multi-criteria evaluation: Methodological foundations and operational consequences," European Journal of Operational Research, Elsevier, vol. 158(3), pages 662-677, November.
    7. Levy, D.T. & Cho, S.-I. & Kim, Y.-M. & Park, S. & Suh, M.-K. & Kam, S., 2010. "SimSmoke model evaluation of the effect of tobacco control policies in Korea: The unknown success story," American Journal of Public Health, American Public Health Association, vol. 100(7), pages 1267-1273.
    8. Ian Sanderson, 2009. "Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning," Political Studies, Political Studies Association, vol. 57(4), pages 699-719, December.
    9. anonymous, 1998. "Evaluating the power of monetary policy," Economics Update, Federal Reserve Bank of Atlanta, issue Oct, pages 1-5.
    10. Lessard, Chantale, 2007. "Complexity and reflexivity: Two important issues for economic evaluation in health care," Social Science & Medicine, Elsevier, vol. 64(8), pages 1754-1765, April.
    11. Brousselle, Astrid & Lessard, Chantale, 2011. "Economic evaluation to inform health care decision-making: Promise, pitfalls and a proposal for an alternative path," Social Science & Medicine, Elsevier, vol. 72(6), pages 832-839, March.
    12. Ian Sanderson, 2009. "Intelligent Policy Making for a Complex World: Pragmatism, Evidence and Learning," Political Studies, Political Studies Association, vol. 57, pages 699-719, December.
    13. Trenholm, Susan & Ferlie, Ewan, 2013. "Using complexity theory to analyse the organisational response to resurgent tuberculosis across London," Social Science & Medicine, Elsevier, vol. 93(C), pages 229-237.
    14. Rametsteiner, Ewald & Weiss, Gerhard, 2006. "Assessing policies from a systems perspecitve -- Experiences with applied innovation systems analysis and implications for policy evaluation," Forest Policy and Economics, Elsevier, vol. 8(5), pages 564-576, July.
    15. -, 1998. "Evaluación de proyectos: tres artículos," Sede de la CEPAL en Santiago (Estudios e Investigaciones) 31002, Naciones Unidas Comisión Económica para América Latina y el Caribe (CEPAL).
    16. Brice Dattée & James Barlow, 2010. "Complexity and whole-system change programmes," Post-Print hal-02312752, HAL.
    17. John G. Burgoyne, 2010. "Evaluating action learning: a critical realist complex network theory approach," Action Learning: Research and Practice, Taylor & Francis Journals, vol. 7(3), pages 239-251, August.
    18. Philip Haynes, 2008. "Complexity Theory and Evaluation in Public Management," Public Management Review, Taylor & Francis Journals, vol. 10(3), pages 401-419, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Moore, Travis & Bakken, Lori L. & Wallace, Bakari, 2019. "The importance of systems thinking, context, and inclusion when studying the feasibility of expanding the FoodShare (SNAP) outreach program to rural communities of Northern Wisconsin," Evaluation and Program Planning, Elsevier, vol. 74(C), pages 10-17.
    2. McGill, Elizabeth & Er, Vanessa & Penney, Tarra & Egan, Matt & White, Martin & Meier, Petra & Whitehead, Margaret & Lock, Karen & Anderson de Cuevas, Rachel & Smith, Richard & Savona, Natalie & Rutter, 2021. "Evaluation of public health interventions from a complex systems perspective: A research methods review," Social Science & Medicine, Elsevier, vol. 272(C).
    3. Michael Walton, 2023. "Adaptive Evaluation: A Complexity-Based Approach to Systematic Learning for Innovation and Scaling in Development," CID Working Papers 428, Center for International Development at Harvard University.
    4. Wilma E. Waterlander & Angie Luna Pinzon & Arnoud Verhoeff & Karen den Hertog & Teatske Altenburg & Coosje Dijkstra & Jutka Halberstadt & Roel Hermans & Carry Renders & Jacob Seidell & Amika Singh & M, 2020. "A System Dynamics and Participatory Action Research Approach to Promote Healthy Living and a Healthy Weight among 10–14-Year-Old Adolescents in Amsterdam: The LIKE Programme," IJERPH, MDPI, vol. 17(14), pages 1-18, July.
    5. Lorthios-Guilledroit, Agathe & Richard, Lucie & Filiatrault, Johanne, 2018. "Factors associated with the implementation of community-based peer-led health promotion programs: A scoping review," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 19-33.
    6. Sarah Chapman & Adiilah Boodhoo & Carren Duffy & Suki Goodman & Maria Michalopoulou, 2023. "Theory of Change in Complex Research for Development Programmes: Challenges and Solutions from the Global Challenges Research Fund," The European Journal of Development Research, Palgrave Macmillan;European Association of Development Research and Training Institutes (EADI), vol. 35(2), pages 298-322, April.
    7. Anna Matheson, 2020. "Health Inequality as a Large-Scale Outcome of Complex Social Systems: Lessons for Action on the Sustainable Development Goals," IJERPH, MDPI, vol. 17(8), pages 1-11, April.
    8. Wei Sun & Alisher Tohirovich Dedahanov & Ho Young Shin & Wei Ping Li, 2021. "Using extended complexity theory to test SMEs’ adoption of Blockchain-based loan system," PLOS ONE, Public Library of Science, vol. 16(2), pages 1-19, February.
    9. Pelucha, Martin & Kveton, Viktor & Potluka, Oto, 2019. "Using mixed method approach in measuring effects of training in firms: Case study of the European Social Fund support," Evaluation and Program Planning, Elsevier, vol. 73(C), pages 146-155.
    10. Gates, Emily F., 2016. "Making sense of the emerging conversation in evaluation about systems thinking and complexity science," Evaluation and Program Planning, Elsevier, vol. 59(C), pages 62-73.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gates, Emily F., 2016. "Making sense of the emerging conversation in evaluation about systems thinking and complexity science," Evaluation and Program Planning, Elsevier, vol. 59(C), pages 62-73.
    2. Anzoise, Valentina & Sardo, Stefania, 2016. "Dynamic systems and the role of evaluation: The case of the Green Communities project," Evaluation and Program Planning, Elsevier, vol. 54(C), pages 162-172.
    3. Jabeen, Sumera, 2016. "Do we really care about unintended outcomes? An analysis of evaluation theory and practice," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 144-154.
    4. Jolley, Gwyneth, 2014. "Evaluating complex community-based health promotion: Addressing the challenges," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 71-81.
    5. Yi Yang, 2021. "Critical realism and complexity theory: Building a nonconstructivist systems research framework for effective governance analysis," Systems Research and Behavioral Science, Wiley Blackwell, vol. 38(1), pages 177-183, January.
    6. Yerko Rojas, 2017. "Evictions and short-term all-cause mortality: a 3-year follow-up study of a middle-aged Swedish population," International Journal of Public Health, Springer;Swiss School of Public Health (SSPH+), vol. 62(3), pages 343-351, April.
    7. O’Connor John, 2022. "Strengthening the science–policy interface in Ireland," Administration, Sciendo, vol. 70(4), pages 29-52, December.
    8. Crabolu, Gloria & Font, Xavier & Eker, Sibel, 2023. "Evaluating policy complexity with Causal Loop Diagrams," Annals of Tourism Research, Elsevier, vol. 100(C).
    9. Azzam, Tarek & Levine, Bret, 2015. "Politics in evaluation: Politically responsive evaluation in high stakes environments," Evaluation and Program Planning, Elsevier, vol. 53(C), pages 44-56.
    10. Ansell, Christopher K. & Bartenberger, Martin, 2016. "Varieties of experimentalism," Ecological Economics, Elsevier, vol. 130(C), pages 64-73.
    11. Erik Nord & Jose Luis Pinto & Jeff Richardson & Paul Menzel & Peter Ubel, 1999. "Incorporating societal concerns for fairness in numerical valuations of health programmes," Health Economics, John Wiley & Sons, Ltd., vol. 8(1), pages 25-39, February.
    12. Claire A Dunlop, 2014. "The Possible Experts: How Epistemic Communities Negotiate Barriers to Knowledge Use in Ecosystems Services Policy," Environment and Planning C, , vol. 32(2), pages 208-228, April.
    13. Srinivasa Vittal Katikireddi & Shona Hilton & Chris Bonell & Lyndal Bond, 2014. "Understanding the Development of Minimum Unit Pricing of Alcohol in Scotland: A Qualitative Study of the Policy Process," PLOS ONE, Public Library of Science, vol. 9(3), pages 1-10, March.
    14. Jessica H. Phoenix & Lucy G. Atkinson & Hannah Baker, 2019. "Creating and communicating social research for policymakers in government," Palgrave Communications, Palgrave Macmillan, vol. 5(1), pages 1-11, December.
    15. Greenhalgh, Trisha & Engebretsen, Eivind, 2022. "The science-policy relationship in times of crisis: An urgent call for a pragmatist turn," Social Science & Medicine, Elsevier, vol. 306(C).
    16. Stucki, Iris, 2018. "Evidence-based arguments in direct democracy: The case of smoking bans in Switzerland," Evaluation and Program Planning, Elsevier, vol. 69(C), pages 148-156.
    17. Deas, L. & Mattu, L. & Gnich, W., 2013. "Intelligent policy making? Key actors' perspectives on the development and implementation of an early years' initiative in Scotland's public health arena," Social Science & Medicine, Elsevier, vol. 96(C), pages 1-8.
    18. Pierre-Olivier Bédard, 2015. "The Mobilization of Scientific Evidence by Public Policy Analysts," SAGE Open, , vol. 5(3), pages 21582440156, September.
    19. Nguyen, Sun V. & Langston, Nancy & Wellstead, Adam & Howlett, Michael, 2020. "Mining the evidence: Public comments and evidence-based policymaking in the controversial Minnesota PolyMet mining project," Resources Policy, Elsevier, vol. 69(C).
    20. William Ascher, 2021. "Coping with intelligence deficits in poverty-alleviation policies in low-income countries," Policy Sciences, Springer;Society of Policy Sciences, vol. 54(2), pages 345-370, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:45:y:2014:i:c:p:119-126. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.