IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v28y2019i1p37-50..html
   My bibliography  Save this article

Formative, embedded evaluation to strengthen interdisciplinary team science: Results of a 4-year, mixed methods, multi-country case study

Author

Listed:
  • Susan Roelofs
  • Nancy Edwards
  • Sarah Viehbeck
  • Cody Anderson

Abstract

Evaluation of interdisciplinary, team science research initiatives is an evolving and challenging field. This descriptive, longitudinal, mixed methods case study examined how an embedded, formative evaluation approach contributed to team science in the interdisciplinary Research into Policy to Enhance Physical Activity (REPOPA) project, which focused on physical activity policymaking in six European countries with divergent policy systems and researcher–policymaker networks. We assessed internal project collaboration, communication, and networking in four annual data collection cycles with REPOPA team members. Data were collected using work package team and individual interviews, and quantitative collaboration and social network questionnaires. Interviews were content analyzed; social networks among team members and with external stakeholder were examined; collaboration scores were compared across 4 years using analysis of variance (ANOVA). Annual monitoring reports with action recommendations were prepared and discussed with consortium members. Results revealed consistently high response rates. Collaboration and communication scores, high at baseline, improved slightly, but ANOVA results were nonsignificant. Internal network changes tracked closely with implementation progress. External stakeholders were primarily governmental, with a marked shift from local/provincial level to national/international during the project. Diversity (disciplinary, organizational, and geopolitical) was a project asset influencing and also challenging collaboration, implementation, and knowledge translation strategies. In conclusion, formative evaluation using an embedded, participatory approach demonstrated utility, acceptability, and researcher engagement. A trusting relationship between evaluators and other project members built on joint identification of team science objectives for the evaluation at project outset, codeveloping guiding principles, and encouraging team reflexivity throughout the evaluation.

Suggested Citation

  • Susan Roelofs & Nancy Edwards & Sarah Viehbeck & Cody Anderson, 2019. "Formative, embedded evaluation to strengthen interdisciplinary team science: Results of a 4-year, mixed methods, multi-country case study," Research Evaluation, Oxford University Press, vol. 28(1), pages 37-50.
    • Handle: RePEc:oup:rseval:v:28:y:2019:i:1:p:37-50.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvy023
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sjoerd Hardeman & Koen Frenken & Önder Nomaler & Anne L. J. Ter Wal, 2015. "Characterizing and comparing innovation systems by different ‘modes’ of knowledge production: A proximity approach," Science and Public Policy, Oxford University Press, vol. 42(4), pages 530-548.
    2. Mila Davids & Koen Frenken, 2018. "Proximity, knowledge base and the innovation process: towards an integrated framework," Regional Studies, Taylor & Francis Journals, vol. 52(1), pages 23-34, January.
    3. Brian M. Belcher & Katherine E. Rasmussen & Matthew R. Kemshaw & Deborah A. Zornes, 2016. "Defining and assessing research quality in a transdisciplinary context," Research Evaluation, Oxford University Press, vol. 25(1), pages 1-17.
    4. Chin-Chang Tsai & Elizabeth A. Corley & Barry Bozeman, 2016. "Collaboration experiences across scientific disciplines and cohorts," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 505-529, August.
    5. Flowers, Alice B., 2010. "Blazing an evaluation pathway: Lessons learned from applying utilization-focused evaluation to a conservation education program," Evaluation and Program Planning, Elsevier, vol. 33(2), pages 165-171, May.
    6. Arsev U. Aydinoglu & Suzie Allard & Chad Mitchell, 2016. "Measuring diversity in disciplinary collaboration in research teams: An ecological perspective," Research Evaluation, Oxford University Press, vol. 25(1), pages 18-36.
    7. Tröster, Christian & Mehra, Ajay & van Knippenberg, Daan, 2014. "Structuring for team success: The interactive effects of network structure and cultural diversity on team potency and performance," Organizational Behavior and Human Decision Processes, Elsevier, vol. 124(2), pages 245-255.
    8. Ali Gazni & Mike Thelwall, 2016. "The citation impact of collaboration between top institutions: A temporal analysis," Research Evaluation, Oxford University Press, vol. 25(2), pages 219-229.
    9. Ron Boschma, 2005. "Proximity and Innovation: A Critical Assessment," Regional Studies, Taylor & Francis Journals, vol. 39(1), pages 61-74.
    10. Cummings, Jonathon N. & Kiesler, Sara, 2007. "Coordination costs and project outcomes in multi-university collaborations," Research Policy, Elsevier, vol. 36(10), pages 1620-1634, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Annette Boaz & Robert Borst & Maarten Kok & Alison O’Shea, 2021. "How far does an emphasis on stakeholder engagement and co-production in research present a threat to academic identity and autonomy? A prospective study across five European countries [Systems Thin," Research Evaluation, Oxford University Press, vol. 30(3), pages 361-369.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Pierre-Alexandre Balland & Ron Boschma & Koen Frenken, 2020. "Proximity, Innovation and Networks: A Concise Review and Some Next Steps," Papers in Evolutionary Economic Geography (PEEG) 2019, Utrecht University, Department of Human Geography and Spatial Planning, Group Economic Geography, revised Mar 2020.
    2. Davids, Mila & Frenken, Koen, 2015. "Proximity, knowledge base and the innovation process The case of Unilever’s Becel diet margarine," Papers in Innovation Studies 2015/7, Lund University, CIRCLE - Centre for Innovation Research.
    3. Ana Fernández & Esther Ferrándiz & M. Dolores León, 2021. "Are organizational and economic proximity driving factors of scientific collaboration? Evidence from Spanish universities, 2001–2010," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 579-602, January.
    4. Pecchioli, Bruno & Moroz, David, 2023. "Do geographical appellations provide useful quality signals? The case of Scotch single malt whiskies," Economic Modelling, Elsevier, vol. 124(C).
    5. Hung, Shiu-Wan & Cheng, Min-Jhih & Hou, Chen-En & Chen, Nai-Rong, 2021. "Inclusion in global virtual teams: Exploring non-spatial proximity and knowledge sharing on innovation," Journal of Business Research, Elsevier, vol. 128(C), pages 599-610.
    6. Viktor Květoň & Josef Novotný & Jiří Blažek & David Marek, 2022. "The role of geographic and cognitive proximity in knowledge networks: The case of joint R&D projects," Papers in Regional Science, Wiley Blackwell, vol. 101(2), pages 351-372, April.
    7. Alexis Pokrovsky & Romain Aeberhardt & Sandra Charreire Petit & Damien Talbot, 2020. "Behind the use of medical facilities, the influence of subjective geographic proximity: a quantitative measuring," Post-Print hal-03762824, HAL.
    8. Tobias Koopmann & Maximilian Stubbemann & Matthias Kapa & Michael Paris & Guido Buenstorf & Tom Hanika & Andreas Hotho & Robert Jäschke & Gerd Stumme, 2021. "Proximity dimensions and the emergence of collaboration: a HypTrails study on German AI research," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9847-9868, December.
    9. Wentian Shi & Wenlong Yang & Debin Du, 2020. "The Scientific Cooperation Network of Chinese Scientists and Its Proximity Mechanism," Sustainability, MDPI, vol. 12(2), pages 1-18, January.
    10. Yawen Qin & Xiaozhen Qin & Haohui Chen & Xun Li & Wei Lang, 2021. "Measuring cognitive proximity using semantic analysis: A case study of China's ICT industry," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 6059-6084, July.
    11. A. Fernández & E. Ferrándiz & M. D. León, 2016. "Proximity dimensions and scientific collaboration among academic institutions in Europe: The closer, the better?," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1073-1092, March.
    12. Daniele Mascia & Francesca Pallotti & Federica Angeli, 2017. "Don’t stand so close to me: competitive pressures, proximity and inter-organizational collaboration," Regional Studies, Taylor & Francis Journals, vol. 51(9), pages 1348-1361, September.
    13. Chen, Kaihua & Zhang, Yi & Fu, Xiaolan, 2019. "International research collaboration: An emerging domain of innovation studies?," Research Policy, Elsevier, vol. 48(1), pages 149-168.
    14. Chin-Chang Tsai & Elizabeth A. Corley & Barry Bozeman, 2016. "Collaboration experiences across scientific disciplines and cohorts," Scientometrics, Springer;Akadémiai Kiadó, vol. 108(2), pages 505-529, August.
    15. Nebojša Stojčić, 2021. "Collaborative innovation in emerging innovation systems: Evidence from Central and Eastern Europe," The Journal of Technology Transfer, Springer, vol. 46(2), pages 531-562, April.
    16. Chollet, Barthélemy & Revet, Karine, 2023. "Digging deep or scratching the surface? Contingent innovation outcomes of seeking advice from geographically distant ties," Technological Forecasting and Social Change, Elsevier, vol. 189(C).
    17. Hajibabaei, Anahita & Schiffauerova, Andrea & Ebadi, Ashkan, 2022. "Gender-specific patterns in the artificial intelligence scientific ecosystem," Journal of Informetrics, Elsevier, vol. 16(2).
    18. Jan Ženka & Ondřej Slach & Igor Ivan, 2020. "Spatial Patterns of Knowledge-Intensive Business Services in Cities of Various Sizes, Morphologies and Economies," Sustainability, MDPI, vol. 12(5), pages 1-19, March.
    19. José Manuel López‐Fernández & Mariluz Maté‐Sánchez‐Val & Francisco Manuel Somohano‐Rodriguez, 2021. "The effect of micro‐territorial networks on industrial small and medium enterprises' innovation: A case study in the Spanish region of Cantabria," Papers in Regional Science, Wiley Blackwell, vol. 100(1), pages 51-77, February.
    20. Zhao‐hui Chong & Jia Liu, 2023. "The evolutionary patterns of intercity co‐invention networks in the Greater Pearl River Delta, China: A comparative analysis based on the technological intensity of industry," Growth and Change, Wiley Blackwell, vol. 54(1), pages 260-283, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:28:y:2019:i:1:p:37-50.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.