IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v30y2021i4p500-513..html
   My bibliography  Save this article

Self-evaluating participatory research projects: A content validation of the InSPIRES online impact evaluation tool
[Content Validity and Reliability of Single Items or Questionnaires]

Author

Listed:
  • Florence Gignac
  • Anne-Sophie Gresle
  • Valeria Santoro Lamelas
  • Montserrat Yepes-Baldó
  • Leonardo de la Torre
  • Maria-Jesus Pinazo
  • the InSPIRES Consortium

Abstract

Research projects involving science shops and citizen science in their promotion of participatory approaches are flourishing globally. However, an instrument evaluating the impacts of such approaches at different stages of a participatory research processes has yet to be validated. The InSPIRES H2020 project developed an impact evaluation tool for just this purpose, consisting of 64 items that reflect upon the dimensions of knowledge democracy, citizen-led research, participatory dynamics, transformative change, and integrity. In this article, we seek to test the content validity of this tool and to provide recommendations that can ensure its validity. A panel of nine experts was created to evaluate each item as regards the following three criteria: representativeness, relevance, and clarity. The Aiken’s V and Wilson Score methods were used to assess the tool’s content validity based on the experts’ ratings. Experts’ written comments were also reviewed. At the panel level, 75% of the items were considered satisfactory in relation to each of the three validity criteria. However, at the population level, 72% of the items suggested that parts of the tool were not valid and required revision. The main suggestions from the experts pointed to the need to reformulate items in which the separation between science and society appeared reinforced and to develop more items about the gender perspective of a research project. The revised version of the tool should serve as a well-founded, comprehensive evaluation instrument for on-going and future projects whose goal is to self-reflect and compare participatory research processes.

Suggested Citation

  • Florence Gignac & Anne-Sophie Gresle & Valeria Santoro Lamelas & Montserrat Yepes-Baldó & Leonardo de la Torre & Maria-Jesus Pinazo & the InSPIRES Consortium, 2021. "Self-evaluating participatory research projects: A content validation of the InSPIRES online impact evaluation tool [Content Validity and Reliability of Single Items or Questionnaires]," Research Evaluation, Oxford University Press, vol. 30(4), pages 500-513.
    • Handle: RePEc:oup:rseval:v:30:y:2021:i:4:p:500-513.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvab026
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rau, Henrike & Goggins, Gary & Fahy, Frances, 2018. "From invisibility to impact: Recognising the scientific and societal relevance of interdisciplinary sustainability research," Research Policy, Elsevier, vol. 47(1), pages 266-276.
    2. Katharina Schlierf & Morgan Meyer, 2013. "Situating knowledge intermediation: Insights from science shops and knowledge brokers," Post-Print hal-00850563, HAL.
    3. Katharina Schlierf & Morgan Meyer, 2013. "Situating knowledge intermediation: Insights from science shops and knowledge brokers," Science and Public Policy, Oxford University Press, vol. 40(4), pages 430-441, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Taheri, Mozhdeh & van Geenhuizen, Marina, 2016. "Teams' boundary-spanning capacity at university: Performance of technology projects in commercialization," Technological Forecasting and Social Change, Elsevier, vol. 111(C), pages 31-43.
    2. Mónica Ramos-Mejía & Alejandro Balanzo, 2018. "What It Takes to Lead Sustainability Transitions from the Bottom-Up: Strategic Interactions of Grassroots Ecopreneurs," Sustainability, MDPI, vol. 10(7), pages 1-20, July.
    3. Marina Van Geenhuizen & Pieter Stek, 2015. "Mapping innovation in the global photovoltaic industry: a bibliometric approach to cluster identification and analysis," ERSA conference papers ersa15p697, European Regional Science Association.
    4. Borst, Robert A.J. & Wehrens, Rik & Bal, Roland & Kok, Maarten Olivier, 2022. "From sustainability to sustaining work: What do actors do to sustain knowledge translation platforms?," Social Science & Medicine, Elsevier, vol. 296(C).
    5. Valentina Tudisca & Adriana Valente, 2016. "(English) Design and implementation of an online Delphi study to develop indicators for evidenceinformed policy making (Italiano) Ideazione e implementazione di uno studio Delphi online per lo svilupp," IRPPS Working Papers 88:2016, National Research Council, Institute for Research on Population and Social Policies.
    6. Paul Benneworth, 2017. "The role of research to shape local and global engagement," CHEPS Working Papers 201706, University of Twente, Center for Higher Education Policy Studies (CHEPS).
    7. Wynanda I. Van Enst & Peter P. J. Driessen & Hens A. C. Runhaar, 2017. "Working at the Boundary: An Empirical Study into the Goals and Strategies of Knowledge Brokers in the Field of Environmental Governance in the Netherlands," Sustainability, MDPI, vol. 9(11), pages 1-14, October.
    8. Lhoste, Evelyne F., 2020. "Can do-it-yourself laboratories open up the science, technology, and innovation research system to civil society?," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    9. Miriam Dunn & Mark D. Rounsevell & Henrik Carlsen & Adis Dzebo & Tiago Capela Lourenço & Joseph Hagg, 2017. "To what extent are land resource managers preparing for high-end climate change in Scotland?," Climatic Change, Springer, vol. 141(2), pages 181-195, March.
    10. de Almeida, Liliane & Augusto de Jesus Pacheco, Diego & Caten, Carla Schwengber ten & Jung, Carlos Fernando, 2021. "A methodology for identifying results and impacts in technological innovation projects," Technology in Society, Elsevier, vol. 66(C).
    11. Dominic Villeneuve & David Durán-Rodas & Anthony Ferri & Tobias Kuttler & Julie Magelund & Michael Mögele & Luca Nitschke & Eriketti Servou & Cat Silva, 2019. "What is Interdisciplinarity in Practice? Critical Reflections on Doing Mobility Research in an Intended Interdisciplinary Doctoral Research Group," Sustainability, MDPI, vol. 12(1), pages 1-20, December.
    12. Jesús de Frutos-Belizón & Fernando Martín-Alcázar & Gonzalo Sánchez-Gardey, 2021. "The research–practice gap in the field of HRM: a qualitative study from the academic side of the gap," Review of Managerial Science, Springer, vol. 15(6), pages 1465-1515, August.
    13. Heikki Tuomenvirta & Hilppa Gregow & Atte Harjanne & Sanna Luhtala & Antti Mäkelä & Karoliina Pilli-Sihvola & Sirkku Juhola & Mikael Hildén & Pirjo Peltonen-Sainio & Ilkka T. Miettinen & Mikko Halonen, 2019. "Identifying Policy Actions Supporting Weather-Related Risk Management and Climate Change Adaptation in Finland," Sustainability, MDPI, vol. 11(13), pages 1-15, July.
    14. De Silva, Muthu & Gokhberg, Leonid & Meissner, Dirk & Russo, Margherita, 2021. "Addressing societal challenges through the simultaneous generation of social and business values: A conceptual framework for science-based co-creation," Technovation, Elsevier, vol. 104(C).
    15. Miltos Ladikas & Julia Hahn & Lei Huang, 2022. "Assessing the Impact of Technology Assessment, Responsible Research and Innovation and Sustainability Research: Towards a Common Methodological Approach," Sustainability, MDPI, vol. 14(4), pages 1-16, February.
    16. Isti Hidayati & Claudia Yamu & Wendy Tan, 2019. "The Emergence of Mobility Inequality in Greater Jakarta, Indonesia: A Socio-Spatial Analysis of Path Dependencies in Transport–Land Use Policies," Sustainability, MDPI, vol. 11(18), pages 1-18, September.
    17. Laura Kreiling & Ahmed Bounfour, 2020. "A practice-based maturity model for holistic TTO performance management: development and initial use," The Journal of Technology Transfer, Springer, vol. 45(6), pages 1718-1747, December.
    18. Keishiro Hara & Iori Miura & Masanori Suzuki & Toshihiro Tanaka, 2023. "Designing research strategy and technology innovation for sustainability by adopting “imaginary future generations”—A case study using metallurgy," Futures & Foresight Science, John Wiley & Sons, vol. 5(3-4), September.
    19. Alba Viana Lora & Marta Gemma Nel-lo Andreu, 2020. "Alternative Metrics for Assessing the Social Impact of Tourism Research," Sustainability, MDPI, vol. 12(10), pages 1-12, May.
    20. Yufeng Chen & Biao Zheng, 2019. "What Happens after the Rare Earth Crisis: A Systematic Literature Review," Sustainability, MDPI, vol. 11(5), pages 1-26, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:30:y:2021:i:4:p:500-513.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.