IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v229y2013i1p143-154.html
   My bibliography  Save this article

Towards a new framework for evaluating systemic problem structuring methods

Author

Listed:
  • Midgley, Gerald
  • Cavana, Robert Y.
  • Brocklesby, John
  • Foote, Jeff L.
  • Wood, David R.R.
  • Ahuriri-Driscoll, Annabel

Abstract

Operational researchers and social scientists often make significant claims for the value of systemic problem structuring and other participative methods. However, when they present evidence to support these claims, it is usually based on single case studies of intervention. There have been very few attempts at evaluating across methods and across interventions undertaken by different people. This is because, in any local intervention, contextual factors, the skills of the researcher and the purposes being pursued by stakeholders affect the perceived success or failure of a method. The use of standard criteria for comparing methods is therefore made problematic by the need to consider what is unique in each intervention. So, is it possible to develop a single evaluation approach that can support both locally meaningful evaluations and longer-term comparisons between methods? This paper outlines a methodological framework for the evaluation of systemic problem structuring methods that seeks to do just this.

Suggested Citation

  • Midgley, Gerald & Cavana, Robert Y. & Brocklesby, John & Foote, Jeff L. & Wood, David R.R. & Ahuriri-Driscoll, Annabel, 2013. "Towards a new framework for evaluating systemic problem structuring methods," European Journal of Operational Research, Elsevier, vol. 229(1), pages 143-154.
  • Handle: RePEc:eee:ejores:v:229:y:2013:i:1:p:143-154
    DOI: 10.1016/j.ejor.2013.01.047
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221713000945
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hjortso, Carsten Nico, 2004. "Enhancing public participation in natural resource management using Soft OR--an application of strategic option development and analysis in tactical forest planning," European Journal of Operational Research, Elsevier, vol. 152(3), pages 667-683, February.
    2. Eden, Colin, 1995. "On evaluating the performance of `wide-band' GDSS's," European Journal of Operational Research, Elsevier, vol. 81(2), pages 302-311, March.
    3. Brocklesby, John, 2009. "Ethics beyond the model: How social dynamics can interfere with ethical practice in operational research/management science," Omega, Elsevier, vol. 37(6), pages 1073-1082, December.
    4. Thomas C. Beierle & David M. Konisky, 2000. "Values, conflict, and trust in participatory environmental planning," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 19(4), pages 587-602.
    5. Valacich, Joseph S. & Schwenk, Charles, 1995. "Devil's Advocacy and Dialectical Inquiry Effects on Face-to-Face and Computer-Mediated Group Decision Making," Organizational Behavior and Human Decision Processes, Elsevier, vol. 63(2), pages 158-173, August.
    6. C. West Churchman, 1970. "Operations Research as a Profession," Management Science, INFORMS, vol. 17(2), pages 37-53, October.
    7. Phahlamohlaka, Jackie & Friend, John, 2004. "Community planning for rural education in South Africa," European Journal of Operational Research, Elsevier, vol. 152(3), pages 684-695, February.
    8. Noah Temaner Jenkins & Michael I. J. Bennett, 1999. "Toward an Empowerment Zone Evaluation," Economic Development Quarterly, , vol. 13(1), pages 23-28, February.
    9. Kelly, K. & Van Vlaenderen, H., 1995. "Evaluating participation processes in community development," Evaluation and Program Planning, Elsevier, vol. 18(4), pages 371-383.
    10. Murphy-Berman, Virginia & Schnoes, Connie & Chambers, Jeff M., 2000. "An early stage evaluation model for assessing the effectiveness of comprehensive community initiatives: three case studies in Nebraska," Evaluation and Program Planning, Elsevier, vol. 23(2), pages 157-163, May.
    11. Pinsonneault, Alain & Kraemer, Kenneth L., 1990. "The effects of electronic meetings on group processes and outcomes: An assessment of the empirical research," European Journal of Operational Research, Elsevier, vol. 46(2), pages 143-161, May.
    12. Steven Yearley, 2006. "Bridging the Science – Policy Divide in Urban Air-Quality Management: Evaluating Ways to Make Models More Robust through Public Engagement," Environment and Planning C, , vol. 24(5), pages 701-714, October.
    13. Masozera, Michel K. & Alavalapati, Janaki R.R. & Jacobson, Susan K. & Shrestha, Ram K., 2006. "Assessing the suitability of community-based management for the Nyungwe Forest Reserve, Rwanda," Forest Policy and Economics, Elsevier, vol. 8(2), pages 206-216, March.
    14. Eden, Colin & Sims, David, 1979. "On the nature of problems in consulting practice," Omega, Elsevier, vol. 7(2), pages 119-127.
    15. Joldersma, Cisca & Roelofs, Ellie, 2004. "The impact of soft OR-methods on problem structuring," European Journal of Operational Research, Elsevier, vol. 152(3), pages 696-708, February.
    16. Alberts, Daniel J., 2007. "Stakeholders or subject matter experts, who should be consulted?," Energy Policy, Elsevier, vol. 35(4), pages 2336-2346, April.
    17. Mingers, John & Brocklesby, John, 1997. "Multimethodology: Towards a framework for mixing methodologies," Omega, Elsevier, vol. 25(5), pages 489-509, October.
    18. Mingers, John & Rosenhead, Jonathan, 2004. "Problem structuring methods in action," European Journal of Operational Research, Elsevier, vol. 152(3), pages 530-554, February.
    19. Bryant, James W. & Darwin, John A., 2004. "Exploring inter-organisational relationships in the health service: An immersive drama approach," European Journal of Operational Research, Elsevier, vol. 152(3), pages 655-666, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pinzon-Salcedo, Luis Arturo & Torres-Cuello, Maria Alejandra, 2018. "Community Operational Research: Developing a systemic peace education programme involving urban and rural communities in Colombia," European Journal of Operational Research, Elsevier, vol. 268(3), pages 946-959.
    2. Howick, Susan & Ackermann, Fran & Walls, Lesley & Quigley, John & Houghton, Tom, 2017. "Learning from mixed OR method practice: The NINES case study," Omega, Elsevier, vol. 69(C), pages 70-81.
    3. Konsti-Laakso, Suvi & Rantala, Tero, 2018. "Managing community engagement: A process model for urban planning," European Journal of Operational Research, Elsevier, vol. 268(3), pages 1040-1049.
    4. Bammer, Gabriele, 2018. "Strengthening community operational research through exchange of tools and strategic alliances," European Journal of Operational Research, Elsevier, vol. 268(3), pages 1168-1177.
    5. White, Leroy, 2018. "A Cook's tour: Towards a framework for measuring the social impact of social purpose organisations," European Journal of Operational Research, Elsevier, vol. 268(3), pages 784-797.
    6. Smith, Chris M. & Shaw, Duncan, 2019. "The characteristics of problem structuring methods: A literature review," European Journal of Operational Research, Elsevier, vol. 274(2), pages 403-416.
    7. Gregory, Amanda J. & Atkins, Jonathan P. & Midgley, Gerald & Hodgson, Anthony M., 2020. "Stakeholder identification and engagement in problem structuring interventions," European Journal of Operational Research, Elsevier, vol. 283(1), pages 321-340.
    8. Herron, Rebecca & Mendiwelso-Bendek, Zoraida, 2018. "Supporting self-organised community research through informal learning," European Journal of Operational Research, Elsevier, vol. 268(3), pages 825-835.
    9. White, Leroy, 2016. "Behavioural operational research: Towards a framework for understanding behaviour in OR interventions," European Journal of Operational Research, Elsevier, vol. 249(3), pages 827-841.
    10. Durugbo, Christopher M., 2020. "Affordance-based problem structuring for workplace innovation," European Journal of Operational Research, Elsevier, vol. 284(2), pages 617-631.
    11. Brocklesby, John & Beall, Elizabeth, 2018. "Processes of engagement and methodology design in Community Operational Research – Insights from the indigenous peoples sector," European Journal of Operational Research, Elsevier, vol. 268(3), pages 996-1005.
    12. Laouris, Yiannis & Michaelides, Marios, 2018. "Structured Democratic Dialogue: An application of a mathematical problem structuring method to facilitate reforms with local authorities in Cyprus," European Journal of Operational Research, Elsevier, vol. 268(3), pages 918-931.
    13. White, Leroy & Burger, Katharina & Yearworth, Mike, 2016. "Understanding behaviour in problem structuring methods interventions with activity theory," European Journal of Operational Research, Elsevier, vol. 249(3), pages 983-1004.
    14. Shaw, Duncan & Smith, Chris M. & Scully, Judy, 2017. "Why did Brexit happen? Using causal mapping to analyse secondary, longitudinal data," European Journal of Operational Research, Elsevier, vol. 263(3), pages 1019-1032.
    15. Yearworth, Mike & White, Leroy, 2014. "The non-codified use of problem structuring methods and the need for a generic constitutive definition," European Journal of Operational Research, Elsevier, vol. 237(3), pages 932-945.
    16. Tavella, Elena & Papadopoulos, Thanos, 2017. "Applying OR to problem situations within community organisations: A case in a Danish non-profit, member-driven food cooperative," European Journal of Operational Research, Elsevier, vol. 258(2), pages 726-742.
    17. Lami, Isabella M. & Tavella, Elena, 2019. "On the usefulness of soft OR models in decision making: A comparison of Problem Structuring Methods supported and self-organized workshops," European Journal of Operational Research, Elsevier, vol. 275(3), pages 1020-1036.
    18. Small, Adrian & Wainwright, David, 2018. "Privacy and security of electronic patient records – Tailoring multimethodology to explore the socio-political problems associated with Role Based Access Control systems," European Journal of Operational Research, Elsevier, vol. 265(1), pages 344-360.
    19. Ufua, Daniel E. & Papadopoulos, Thanos & Midgley, Gerald, 2018. "Systemic Lean Intervention: Enhancing Lean with Community Operational Research," European Journal of Operational Research, Elsevier, vol. 268(3), pages 1134-1148.
    20. Luoma, Jukka, 2016. "Model-based organizational decision making: A behavioral lens," European Journal of Operational Research, Elsevier, vol. 249(3), pages 816-826.
    21. Cronin, Karen & Midgley, Gerald & Jackson, Laurie Skuba, 2014. "Issues Mapping: A problem structuring method for addressing science and technology conflicts," European Journal of Operational Research, Elsevier, vol. 233(1), pages 145-158.
    22. van Antwerpen, Coen & Curtis, Neville J., 2016. "A data collection and presentation methodology for decision support: A case study of hand-held mine detection devices," European Journal of Operational Research, Elsevier, vol. 251(1), pages 237-251.
    23. Gregory, Amanda J. & Atkins, Jonathan P., 2018. "Community Operational Research and Citizen Science: Two icons in need of each other?," European Journal of Operational Research, Elsevier, vol. 268(3), pages 1111-1124.
    24. Brocklesby, John & Midgley, Gerald, 2016. "Boundary games: How teams of OR practitioners explore the boundaries of interventionAuthor-Name: Velez-Castiblanco, Jorge," European Journal of Operational Research, Elsevier, vol. 249(3), pages 968-982.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:229:y:2013:i:1:p:143-154. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Haili He). General contact details of provider: http://www.elsevier.com/locate/eor .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.