IDEAS home Printed from https://ideas.repec.org/p/hal/journl/hal-02167827.html
   My bibliography  Save this paper

The BETA-EvaRIO impact evaluation method: towards a bridging approach?

Author

Listed:
  • Laurent Bach

    (BETA - Bureau d'Économie Théorique et Appliquée - INRA - Institut National de la Recherche Agronomique - UNISTRA - Université de Strasbourg - UL - Université de Lorraine - CNRS - Centre National de la Recherche Scientifique)

  • Sandrine Wolff

    (BETA - Bureau d'Économie Théorique et Appliquée - INRA - Institut National de la Recherche Agronomique - UNISTRA - Université de Strasbourg - UL - Université de Lorraine - CNRS - Centre National de la Recherche Scientifique)

Abstract

This paper presents a specific approach in the field of evaluating the effects of publicly supported R&D activities: the BETA approach. Initially developed for application to large technology procurement/agency driven public R&D programmes, it has been adapted to a larger range of programmes supporting science and technology. The BETA approach consists of identifying and retrospectively measuring, at the micro-level via direct interviews, the impact of different types of learning processes triggered by participation in R&D. A further adaptation to the specific case of research infrastructures was conducted in the frame of the EvaRIO project. It also dealt with learning processes from an ex post and micro-level perspective, but covered a wider variety of activities and actors over time, leading to a broader view of effects. Based on 30 years of impact evaluation experience, we argue that the revised BETA approach, despite its focus on the economic dimension of impacts, is at the intersection between different strands of evaluation research. The dual nature of the BETA approach is examined along several dimensions including an output versus process perspective, project versus organization scope, qualitative versus quantitative investigation, contribution versus attribution issues, private profitability versus public value perspective, etc. We show that the BETA-EvaRIO method can be used to complement other evaluation methodologies and can be considered as bridging among them, or can be seen as a single, specific approach.

Suggested Citation

  • Laurent Bach & Sandrine Wolff, 2017. "The BETA-EvaRIO impact evaluation method: towards a bridging approach?," Post-Print hal-02167827, HAL.
  • Handle: RePEc:hal:journl:hal-02167827
    DOI: 10.1007/s10961-017-9603-y
    Note: View the original document on HAL open archive server: https://hal.univ-lorraine.fr/hal-02167827
    as

    Download full text from publisher

    File URL: https://hal.univ-lorraine.fr/hal-02167827/document
    Download Restriction: no

    File URL: https://libkey.io/10.1007/s10961-017-9603-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Chiara PANCOTTI & Julie PELLEGRIN & Silvia VIGNETTI, 2014. "Appraisal of Research Infrastructures: Approaches, methods and practical implications," Departmental Working Papers 2014-13, Department of Economics, Management and Quantitative Methods at Università degli Studi di Milano.
    2. Abdullah Gök & Jakob Edler, 2012. "The use of behavioural additionality evaluation in innovation policy making," Research Evaluation, Oxford University Press, vol. 21(4), pages 306-318, September.
    3. Gaunand, A. & Hocdé, A. & Lemarié, S. & Matt, M. & Turckheim, E.de, 2015. "How does public agricultural research impact society? A characterization of various patterns," Research Policy, Elsevier, vol. 44(4), pages 849-861.
    4. Laurent Bach & Mireille Matt, 2005. "From Economic Foundations to S&T Policy Tools: a Comparative Analysis of the Dominant Paradigms," Springer Books, in: Patrick Llerena & Mireille Matt (ed.), Innovation Policy in a Knowledge-Based Economy, chapter 1, pages 17-45, Springer.
    5. Autio, Erkko & Hameri, Ari-Pekka & Vuola, Olli, 2004. "A framework of industrial knowledge spillovers in big-science centers," Research Policy, Elsevier, vol. 33(1), pages 107-126, January.
    6. Albert N. Link & Nicholas S. Vonortas (ed.), 2013. "Handbook on the Theory and Practice of Program Evaluation," Books, Edward Elgar Publishing, number 14384.
    7. Laurent Bach & Mireille Matt, 2005. "Twenty Years of Evaluation with the BETA Method : Some Insights on Current Collaborative ST&I Policy Issues," Post-Print hal-00279441, HAL.
    8. Laurent Bach & Mireille Matt, 2005. "From Economic Foundations to S&T Policy Tools : A Comparative Analysis of the Dominant Paradigms," Post-Print hal-00279442, HAL.
    9. David J. Teece & Gary Pisano & Amy Shuen, 1997. "Dynamic capabilities and strategic management," Strategic Management Journal, Wiley Blackwell, vol. 18(7), pages 509-533, August.
    10. Toole, Andrew A., 2012. "The impact of public basic research on industrial innovation: Evidence from the pharmaceutical industry," Research Policy, Elsevier, vol. 41(1), pages 1-12.
    11. Olof Hallonsten, 2013. "Introducing ‘facilitymetrics’: a first review and analysis of commonly used measures of scientific leadership among synchrotron radiation facilities worldwide," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 497-513, August.
    12. Claire Donovan, 2011. "State of the art in assessing research impact: introduction to a special issue," Research Evaluation, Oxford University Press, vol. 20(3), pages 175-179, September.
    13. Laurent Bach & Mireille Matt, 2005. "Twenty Years of Evaluation with the BETA Method: Some Insights on Current Collaborative ST&I Policy Issues," Springer Books, in: Patrick Llerena & Mireille Matt (ed.), Innovation Policy in a Knowledge-Based Economy, chapter 9, pages 251-281, Springer.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nathalie Taverdet-Popiolek, 2022. "Economic Footprint of a Large French Research and Technology Organisation in Europe: Deciphering a Simplified Model and Appraising the Results," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 44-69, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laurent Bach & Sandrine Wolff, 2022. "The BETA-EvaRIO impact evaluation method: towards a bridging approach?," The Journal of Technology Transfer, Springer, vol. 47(3), pages 651-672, June.
    2. Kiman Kim & Sang Ok Choi & Sooyeon Lee, 2021. "The Effect of a Financial Support on Firm Innovation Collaboration and Output: Does Policy Work on the Diverse Nature of Firm Innovation?," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 12(2), pages 645-675, June.
    3. Federica Rossi & Annalisa Caloffi & Margherita Russo, 2013. "Networked by design: Can policy constraints support the development of capabilities for collaborative innovation?," Management Working Papers 5, Birkbeck Department of Management, revised Jul 2013.
    4. Rossi, Federica & Caloffi, Annalisa & Russo, Margherita, 2016. "Networked by design: Can policy requirements influence organisations' networking behaviour?," Technological Forecasting and Social Change, Elsevier, vol. 105(C), pages 203-214.
    5. Bleda, Mercedes & del Río, Pablo, 2013. "The market failure and the systemic failure rationales in technological innovation systems," Research Policy, Elsevier, vol. 42(5), pages 1039-1052.
    6. Attila Havas, 2014. "Trapped by the high-tech myth: the need and chances for a new policy rationale," Chapters, in: Hartmut Hirsch-Kreinsen & Isabel Schwinge (ed.), Knowledge-Intensive Entrepreneurship in Low-Tech Industries, chapter 9, pages 193-217, Edward Elgar Publishing.
    7. Attila Havas, 2015. "The persistent high-tech myth in the EC policy circles - Implications for the EU10 countries," CERS-IE WORKING PAPERS 1517, Institute of Economics, Centre for Economic and Regional Studies.
    8. Irwin Feller, 2022. "Assessing the societal impact of publicly funded research," The Journal of Technology Transfer, Springer, vol. 47(3), pages 632-650, June.
    9. Havas, Attila, 2014. "Mit mér(j)ünk?. Az innováció értelmezései - szakpolitikai következmények [The theory and measurement of innovation and its mutual effect on policy]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(9), pages 1022-1059.
    10. Andrea, Bastianin & Paolo, Castelnuovo & Massimo, Florio & Anna, Giunta, 2019. "Technological Learning and Innovation Gestation Lags at the Frontier of Science: from CERN Procurement to Patents," Working Papers 405, University of Milano-Bicocca, Department of Economics, revised Apr 2019.
    11. Joanna Stryjek, 2021. "Counteracting the COVID-19 Crisis with Innovation Policy Tools: A Case Study of the EU’s Supranational Innovation Policy," European Research Studies Journal, European Research Studies Journal, vol. 0(3), pages 450-468.
    12. Matt, M. & Colinet, L. & Gaunand, A. & Joly, P.B., 2015. "A typology of impact pathways generated by a public agricultural research organization," Working Papers 2015-03, Grenoble Applied Economics Laboratory (GAEL).
    13. Pekka SALMI, 2012. "The Impact of Public R&D Funding on Open Innovation," Economia. Seria Management, Faculty of Management, Academy of Economic Studies, Bucharest, Romania, vol. 15(1), pages 142-163, June.
    14. Nils Grashof, 2019. "Firm-specific cluster effects - A meta-analysis," Bremen Papers on Economics & Innovation 1906, University of Bremen, Faculty of Business Studies and Economics.
    15. Lutz Bornmann, 2013. "What is societal impact of research and how can it be assessed? a literature survey," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(2), pages 217-233, February.
    16. Jakob Edler & Jan Fagerberg, 2017. "Innovation policy: what, why, and how," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 33(1), pages 2-23.
    17. Turner, James A & Guesmi, Bouali & Gil, José M. & Heanue, Kevin & Sierra, Miguel & Percy, Helen & Bortagaray, Isabel & Chams, Nour & Milne, Cath, 2022. "Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay," Evaluation and Program Planning, Elsevier, vol. 94(C).
    18. Federico Frattini, 2011. "Mature Systems in Global Markets: The Need for an Institutional Division of Labour," Working Papers 201109, University of Ferrara, Department of Economics.
    19. Peter Weißhuhn & Katharina Helming & Johanna Ferretti, 2018. "Research impact assessment in agriculture—A review of approaches and impact areas," Research Evaluation, Oxford University Press, vol. 27(1), pages 36-42.
    20. Simachev, Yuri & Kuzyk, Mikhail & Feygina, Vera, 2015. "Interaction between Business and Research Organizations in the Sphere of Innovations: The Russian Experience in Promoting Cooperation," Published Papers 431503, Russian Presidential Academy of National Economy and Public Administration.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:journl:hal-02167827. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.