IDEAS home Printed from https://ideas.repec.org/a/sae/sagope/v6y2016i3p2158244016663800.html
   My bibliography  Save this article

Developing a Framework for Research Evaluation in Complex Contexts Such as Action Research

Author

Listed:
  • Eileen Piggot-Irvine
  • Deborah Zornes

Abstract

Early investigation led the Evaluative Study of Action Research (ESAR) team to conclude that the complexity of a global, large scale study (evaluation of more than 100 highly diverse action research [AR] projects) called for an overarching research evaluation framework that differed from traditional frameworks. This article details the flexible, rigorous, Evaluative Action Research (EvAR) framework developed to meet the complex demands of the diverse AR projects and the intent to conduct high engagement research evaluation. The EvAR fulfilled multiple overarching needs to: authentically collaborate, engage, and enhance ownership from the ESAR team and the AR project participants and boundary partners evaluated; be informed in decision making via strong reference support; be responsive and flexible yet meet accountability demands to track, demonstrate, and measure process, outcomes, and impacts of projects; use mixed-method data collection to enhance rigor of findings; and utilize a highly reflective and reflexive approach to the evaluation. Many of the latter needs align with underpinning principles and values in AR itself; that is, it is collaborative, consultative, democratic, reflective, reflexive, dialogical, and improvement oriented. Rationale for the framework is provided alongside full details of phases and implementation elements using the ESAR as an example. Throughout the article, features are highlighted that distinguish this new EvAR framework from others. The advantages of adopting a flexible framework, which aims to enhance engagement of those evaluated, are highly relevant to contexts beyond AR if ownership of evaluation outcomes is a goal.

Suggested Citation

  • Eileen Piggot-Irvine & Deborah Zornes, 2016. "Developing a Framework for Research Evaluation in Complex Contexts Such as Action Research," SAGE Open, , vol. 6(3), pages 21582440166, August.
    • Handle: RePEc:sae:sagope:v:6:y:2016:i:3:p:2158244016663800
    DOI: 10.1177/2158244016663800
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2158244016663800
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2158244016663800?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Julie Thompson Klein, 2006. "Afterword: the emergent literature on interdisciplinary and transdisciplinary research evaluation," Research Evaluation, Oxford University Press, vol. 15(1), pages 75-80, April.
    2. Charles Conteh, 2013. "Strategic Inter-Organizational Cooperation in Complex Environments," Public Management Review, Taylor & Francis Journals, vol. 15(4), pages 501-521, May.
    3. Claire Donovan & Stephen Hanney, 2011. "The ‘Payback Framework’ explained," Research Evaluation, Oxford University Press, vol. 20(3), pages 181-183, September.
    4. Stefan P L de Jong & Pleun van Arensbergen & Floortje Daemen & Barend van der Meulen & Peter van den Besselaar, 2011. "Evaluation of research in context: an approach and two cases," Research Evaluation, Oxford University Press, vol. 20(1), pages 61-72, March.
    5. Tremblay, Crystal & Hall, Budd L., 2014. "Learning from community-university research partnerships: A Canadian study on community impact and conditions for success," International Journal of Action Research, Rainer Hampp Verlag, vol. 10(3), pages 376-404.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cian O’Donovan & Aleksandra (Ola) Michalec & Joshua R Moon, 2022. "Capabilities for transdisciplinary research," Research Evaluation, Oxford University Press, vol. 31(1), pages 145-158.
    2. de Jong, Stefan P.L. & Wardenaar, Tjerk & Horlings, Edwin, 2016. "Exploring the promises of transdisciplinary research: A quantitative study of two climate research programmes," Research Policy, Elsevier, vol. 45(7), pages 1397-1409.
    3. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    4. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    5. Adriana Zaiț & Constantin Bratianu & Elena‐Mădălina Vătămănescu & Andreia Gabriela Andrei & Ioana Alexandra Horodnic, 2021. "Interdisciplinarity: A complexity approach towards academic research," Systems Research and Behavioral Science, Wiley Blackwell, vol. 38(3), pages 294-306, May.
    6. Pettersson, Fredrik & Westerdahl, Stig & Hansson, Joel, 2018. "Learning through collaboration in the Swedish public transport sector? Co-production through guidelines and living labs," Research in Transportation Economics, Elsevier, vol. 69(C), pages 394-401.
    7. Maaike Verbree & Edwin Horlings & Peter Groenewegen & Inge Weijden & Peter Besselaar, 2015. "Organizational factors influencing scholarly performance: a multivariate study of biomedical research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 25-49, January.
    8. Guesmi, B. & Gil, J.M., 2018. "Monitoring and Evaluation of Public Agricultural Research Organization: Impact oriented monitoring approach," 2018 Conference, July 28-August 2, 2018, Vancouver, British Columbia 277554, International Association of Agricultural Economists.
    9. Roberto Lopez-Olmedo & Norma Georgina Gutierrez-Serrano, 2021. "Transdisciplinary knowledge production in mainstream journals: from the perspective of the participation of social actors in Mexico," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(3), pages 2627-2641, March.
    10. Blanca L. Díaz Mariño & Frida Carmina Caballero-Rico & Ramón Ventura Roque Hernández & José Alberto Ramírez de León & Daniel Alejandro González-Bandala, 2021. "Towards the Construction of Productive Interactions for Social Impact," Sustainability, MDPI, vol. 13(2), pages 1-20, January.
    11. Jaroslav Dvorak & Remigijus Civinskas, 2019. "The Sweden Model of Information Exchange and Partnership between Stakeholders: The Case of Posting Workers," Public administration issues, Higher School of Economics, issue 1, pages 190-210.
    12. Gotsche, Caroline I & Weishaar, Heide & Hanefeld, Johanna, 2023. "Global health in Germany: Understanding interdisciplinarity in the academic sector," Health Policy, Elsevier, vol. 130(C).
    13. K. Jonkers & G. E. Derrick & C. Lopez-Illescas & P. Besselaar, 2014. "Measuring the scientific impact of e-research infrastructures: a citation based approach?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1179-1194, November.
    14. Дворак Я. & Цивинскас Р., 2019. "Шведская Модель Обмена Информации И Партнерства Среди Заинтересованных Субъектов: Пример Командированных Работников," Вопросы государственного и муниципального управления // Public administration issues, НИУ ВШЭ, issue 1, pages 190-210.
    15. Robert Hrelja & Fredrik Pettersson & Stig Westerdahl, 2016. "The Qualities Needed for a Successful Collaboration: A Contribution to the Conceptual Understanding of Collaboration for Efficient Public Transport," Sustainability, MDPI, vol. 8(6), pages 1-15, June.
    16. Stefan P L de Jong & Corina Balaban, 2022. "How universities influence societal impact practices: Academics’ sense-making of organizational impact strategies [Between Relevance and Excellence? Research Impact Agenda and the Production of Pol," Science and Public Policy, Oxford University Press, vol. 49(4), pages 609-620.
    17. Pleun Arensbergen & Inge van der Weijden & Peter Besselaar, 2012. "Gender differences in scientific productivity: a persisting phenomenon?," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(3), pages 857-868, December.
    18. van den Hurk, Martijn & Verhoest, Koen, 2017. "On the fast track? Using standard contracts in public–private partnerships for sports facilities: A case study," Sport Management Review, Elsevier, vol. 20(2), pages 226-239.
    19. Dag W. Aksnes & Liv Langfeldt & Paul Wouters, 2019. "Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories," SAGE Open, , vol. 9(1), pages 21582440198, February.
    20. Matteo Pedrini & Valentina Langella & Mario Alberto Battaglia & Paola Zaratin, 2018. "Assessing the health research’s social impact: a systematic review," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1227-1250, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:sagope:v:6:y:2016:i:3:p:2158244016663800. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.