IDEAS home Printed from https://ideas.repec.org/a/eee/respol/v48y2019i1p312-321.html
   My bibliography  Save this article

Towards system oriented innovation policy evaluation? Evidence from EU28 member states

Author

Listed:
  • Borrás, Susana
  • Laatsit, Mart

Abstract

Many years after the introduction of the innovation system concept in innovation policy design, it is still not clear whether innovation policy evaluation practices follow a system approach. Building on evaluation and innovation studies, this article develops the concept ‘system oriented innovation policy evaluation’ based on four attributes (coverage, perspective, temporality and expertise). The attributes are used as analytical devices for gathering extensive empirical evidence on the actual practices of EU28 member states. The findings show that few countries have developed a type of innovation policy evaluation that is system oriented. The advent of a system approach to innovation policy evaluation offers the opportunity of comprehensive, contextualized and evidence-based innovation policy-making. However, there are still serious obstacles as such an approach requires important knowledge and organisational capacities. Overcoming these obstacles would need more decided evaluation capacity-building at the national level.

Suggested Citation

  • Borrás, Susana & Laatsit, Mart, 2019. "Towards system oriented innovation policy evaluation? Evidence from EU28 member states," Research Policy, Elsevier, vol. 48(1), pages 312-321.
    • Handle: RePEc:eee:respol:v:48:y:2019:i:1:p:312-321
    DOI: 10.1016/j.respol.2018.08.020
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0048733318302075
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.respol.2018.08.020?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Martin, Ben R. & Nightingale, Paul & Yegros-Yegros, Alfredo, 2012. "Science and technology studies: Exploring the knowledge base," Research Policy, Elsevier, vol. 41(7), pages 1182-1204.
    2. Jerald Hage & Gretchen Jordan & Jonathan Mote, 2007. "A theory-based innovation systems framework for evaluating diverse portfolios of research, part two: Macro indicators and policy interventions," Science and Public Policy, Oxford University Press, vol. 34(10), pages 731-741, December.
    3. Sartori, Giovanni, 1970. "Concept Misformation in Comparative Politics," American Political Science Review, Cambridge University Press, vol. 64(4), pages 1033-1053, December.
    4. Paul Cunningham & Jakob Edler & Kieron Flanagan & Philippe Larédo, 2016. "The innovation policy mix," Chapters, in: Jakob Edler & Paul Cunningham & Abdullah Gök & Philip Shapira (ed.), Handbook of Innovation Policy Impact, chapter 17, pages 505-542, Edward Elgar Publishing.
    5. Magro, Edurne & Wilson, James R., 2013. "Complex innovation policy systems: Towards an evaluation mix," Research Policy, Elsevier, vol. 42(9), pages 1647-1656.
    6. Jakob Edler & Bernd Ebersberger & Vivien Lo, 2008. "Improving policy understanding by means of secondary analyses of policy evaluation," Research Evaluation, Oxford University Press, vol. 17(3), pages 175-186, September.
    7. Ruud E. Smits & Stefan Kuhlmann & Phillip Shapira (ed.), 2010. "The Theory and Practice of Innovation Policy," Books, Edward Elgar Publishing, number 4181.
    8. Reinhilde Veugelers (ed.), 2009. "The Evaluation of the Finnish National Innovation System - Full Report," Books, The Research Institute of the Finnish Economy, number 495.
    9. Jakob Edler & Paul Cunningham & Abdullah Gök & Philip Shapira (ed.), 2016. "Handbook of Innovation Policy Impact," Books, Edward Elgar Publishing, number 16121.
    10. Stefan Kuhlmann & Philip Shapira & Ruud Smits, 2010. "Introduction. A Systemic Perspective: The Innovation Policy Dance," Chapters, in: Ruud E. Smits & Stefan Kuhlmann & Phillip Shapira (ed.), The Theory and Practice of Innovation Policy, chapter 1, Edward Elgar Publishing.
    11. Erik Arnold, 2004. "Evaluating research and innovation policy: a systems world needs systems evaluations," Research Evaluation, Oxford University Press, vol. 13(1), pages 3-17, April.
    12. Irwin Feller, 2007. "Mapping the frontiers of evaluation of public-sector R&D programs," Science and Public Policy, Oxford University Press, vol. 34(10), pages 681-690, December.
    13. Mari Jose Aranguren & Edurne Magro & James R Wilson, 2017. "Regional competitiveness policy evaluation as a transformative process: From theory to practice," Environment and Planning C, , vol. 35(4), pages 703-720, June.
    14. Karl Aiginger & Rahel Falk & Andreas Reinstaller, 2009. "Evaluation of Government Funding in RTDI from a Systems Perspective in Austria. Synthesis Report," WIFO Studies, WIFO, number 36402, February.
    15. Susana Borrás, 2011. "Policy learning and organizational capacities in innovation policies," Science and Public Policy, Oxford University Press, vol. 38(9), pages 725-734, November.
    16. Flanagan, Kieron & Uyarra, Elvira & Laranja, Manuel, 2011. "Reconceptualising the 'policy mix' for innovation," Research Policy, Elsevier, vol. 40(5), pages 702-713, June.
    17. Jakob Edler & Martin Berger & Michael Dinges & Abdullah Gök, 2012. "The practice of evaluation in innovation policy in Europe," Research Evaluation, Oxford University Press, vol. 21(3), pages 167-182, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Foreman-Peck, James & Zhou, Peng, 2022. "R&D subsidies and productivity in eastern European countries," Economic Systems, Elsevier, vol. 46(2).
    2. Foreman-Peck, James & Zhou, Peng, 2021. "Innovation policy and performance of Eastern European Countries," Cardiff Economics Working Papers E2021/15, Cardiff University, Cardiff Business School, Economics Section.
    3. Maping Zhang & Zongjun Wang & Xue Wang, 2023. "Configuration of Conditions Leading to High National Innovation Competitiveness: A Fuzzy Set Qualitative Comparative Analysis Approach," Sustainability, MDPI, vol. 15(18), pages 1-15, September.
    4. Yang Gao, 2022. "The Belt and Road Initiative and cascading innovation in China’s domestic railway ecosystem," Journal of International Business Policy, Palgrave Macmillan, vol. 5(2), pages 236-258, June.
    5. Ricard Esparza-Masana, 2022. "Towards Smart Specialisation 2.0. Main Challenges When Updating Strategies," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 635-655, March.
    6. Marra, Mita, 2020. "A few introductory notes on the special issue on the Evaluation Framework for Promoting Gender Equality in Research and Innovation (EFFORTI)," Evaluation and Program Planning, Elsevier, vol. 79(C).
    7. Ron Crawford, 2021. "Focused innovation policy: Lessons from international experience," Working Papers 2021/03, New Zealand Productivity Commission.
    8. Laatsit, Mart & Lindholm-Dahlstrand, Åsa & Nilsson, Magnus, 2023. "Capturing the system-level effects of innovation policy: an assessment of publicly funded innovative entrepreneurship in Sweden," Papers in Innovation Studies 2023/12, Lund University, CIRCLE - Centre for Innovation Research.
    9. Matthijs Jansen, 2022. "Legitimation and Effects of Mission-Oriented Innovation Policies: A Spillover Perspective," Hacienda Pública Española / Review of Public Economics, IEF, vol. 243(4), pages 7-28, December.
    10. Josie Coburn & Frederique Bone & Andy C. Stirling & Michael M. Hopkins & Jorge Mestre-Ferrandiz & Stathis Arapostathis & Martin J. Llewelyn, 2021. "Appraising research policy instrument mixes: a multicriteria mapping study in six European countries of diagnostic innovation to manage antimicrobial resistance," SPRU Working Paper Series 2021-03, SPRU - Science Policy Research Unit, University of Sussex Business School.
    11. L Carlos Freire-Gibb & Geoff Gregson, 2019. "Innovation systems and entrepreneurial ecosystems: Implications for policy and practice in Latin America," Local Economy, London South Bank University, vol. 34(8), pages 787-806, December.
    12. Coburn, Josie & Bone, Frederique & Hopkins, Michael M. & Stirling, Andy & Mestre-Ferrandiz, Jorge & Arapostathis, Stathis & Llewelyn, Martin J., 2021. "Appraising research policy instrument mixes: a multicriteria mapping study in six European countries of diagnostic innovation to manage antimicrobial resistance," Research Policy, Elsevier, vol. 50(4).
    13. Kern, Florian & Rogge, Karoline S. & Howlett, Michael, 2019. "Policy mixes for sustainability transitions: New approaches and insights through bridging innovation and policy studies," Research Policy, Elsevier, vol. 48(10).
    14. Song Wang & Jiexin Wang & Chenqi Wei & Xueli Wang & Fei Fan, 2021. "Collaborative innovation efficiency: From within cities to between cities—Empirical analysis based on innovative cities in China," Growth and Change, Wiley Blackwell, vol. 52(3), pages 1330-1360, September.
    15. Reidl, Sybille & Beranek, Sarah & Holzinger, Florian & Streicher, Jürgen, 2020. "Gender equality regimes and evaluation regimes in Europe and their implications for policy design and evaluation," Evaluation and Program Planning, Elsevier, vol. 83(C).
    16. Pintar, Nico & Scherngell, Thomas, 2022. "The complex nature of regional knowledge production: Evidence on European regions," Research Policy, Elsevier, vol. 51(8).
    17. Aalto, Eero & Gustafsson, Robin, 2020. "Innovation Promotion Rationales and Impacts – A Review," ETLA Reports 99, The Research Institute of the Finnish Economy.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jakob Edler & Jan Fagerberg, 2017. "Innovation policy: what, why, and how," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 33(1), pages 2-23.
    2. Ricard Esparza-Masana, 2022. "Towards Smart Specialisation 2.0. Main Challenges When Updating Strategies," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(1), pages 635-655, March.
    3. Matthijs Janssen, 2016. "What bangs for your bucks? Assessing the design and impact of transformative policy," Innovation Studies Utrecht (ISU) working paper series 16-05, Utrecht University, Department of Innovation Studies, revised Dec 2016.
    4. Claire Nauwelaers & Inger Midtkandal & Inmaculada Periañez Forte, 2014. "RIS3 Implementation and Policy Mixes. S3 Policy Brief Series n° 07/2014 September 2014," JRC Research Reports JRC91917, Joint Research Centre.
    5. Janssen, Matthijs J., 2019. "What bangs for your buck? Assessing the design and impact of Dutch transformative policy," Technological Forecasting and Social Change, Elsevier, vol. 138(C), pages 78-94.
    6. Magro, Edurne & Wilson, James R., 2019. "Policy-mix evaluation: Governance challenges from new place-based innovation policies," Research Policy, Elsevier, vol. 48(10).
    7. Magro, Edurne & Wilson, James R., 2013. "Complex innovation policy systems: Towards an evaluation mix," Research Policy, Elsevier, vol. 42(9), pages 1647-1656.
    8. Mari Jose Aranguren & Edurne Magro & James R Wilson, 2017. "Regional competitiveness policy evaluation as a transformative process: From theory to practice," Environment and Planning C, , vol. 35(4), pages 703-720, June.
    9. Denis Gray & Lindsey McGowen & Timothy L. Michaelis & Olena Leonchuk & Drew Rivers, 2022. "A policy mix experiment to promote start-up success: exploratory evaluation of the NSF Small Business Innovation Research (SBIR)/Industry University Cooperative Research Center (IUCRC) membership supp," The Journal of Technology Transfer, Springer, vol. 47(1), pages 176-212, February.
    10. Esparza Masana, Ricard & Fernández, Tatiana, 2019. "Monitoring S3: Key dimensions and implications," Evaluation and Program Planning, Elsevier, vol. 77(C).
    11. Hilde Nykamp, 2020. "Policy Mix for a Transition to Sustainability: Green Buildings in Norway," Sustainability, MDPI, vol. 12(2), pages 1-17, January.
    12. Jesper Lindgaard Christensen & Ina Drejer & Poul Houman Andersen & Jacob Rubæk Holm, 2016. "Innovation policy: how can it best make a difference?," Industry and Innovation, Taylor & Francis Journals, vol. 23(2), pages 135-139, February.
    13. Kergroach, Sandrine, 2019. "National innovation policies for technology upgrading through GVCs: A cross-country comparison," Technological Forecasting and Social Change, Elsevier, vol. 145(C), pages 258-272.
    14. Wise, Emily & Arnold, Erik, 2022. "Evaluating Transformation – what can we learn from the literature?," Papers in Innovation Studies 2022/10, Lund University, CIRCLE - Centre for Innovation Research.
    15. Purkus, Alexandra & Lüdtke, Jan, 2020. "A systemic evaluation framework for a multi-actor, forest-based bioeconomy governance process: The German Charter for Wood 2.0 as a case study," Forest Policy and Economics, Elsevier, vol. 113(C).
    16. Grashof, Nils, 2020. "Putting the watering can away Towards a targeted (problem-oriented) cluster policy framework," Papers in Innovation Studies 2020/4, Lund University, CIRCLE - Centre for Innovation Research.
    17. Valeria Costantini & Francesco Crespi & Alessandro Palma, 2015. "Characterizing the policy mix and its impact on eco-innovation in energy-efficient technologies," SEEDS Working Papers 1115, SEEDS, Sustainability Environmental Economics and Dynamics Studies, revised Jun 2015.
    18. Dirk Meissner & Sandrine Kergroach, 2021. "Innovation policy mix: mapping and measurement," The Journal of Technology Transfer, Springer, vol. 46(1), pages 197-222, February.
    19. Bleda, Mercedes & del Río, Pablo, 2013. "The market failure and the systemic failure rationales in technological innovation systems," Research Policy, Elsevier, vol. 42(5), pages 1039-1052.
    20. Haddad, Carolina R. & Bergek, Anna, 2023. "Towards an integrated framework for evaluating transformative innovation policy," Research Policy, Elsevier, vol. 52(2).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:respol:v:48:y:2019:i:1:p:312-321. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/respol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.