IDEAS home Printed from https://ideas.repec.org/a/oup/rseval/v21y2012i1p33-47.html
   My bibliography  Save this article

Evaluating the demand side: New challenges for evaluation

Author

Listed:
  • J. Edler
  • L. Georghiou
  • K. Blind
  • E. Uyarra

Abstract

Evaluation of research and innovation policy faces radical challenges arising from a new policy emphasis upon demand-side measures and linked to this an understanding of innovation policy as a means to achieve societal goals. This article considers the implications for the practice of evaluation at both micro and meso-levels. It uses the exemplar of an evaluation design for the European Union's Lead Market Initiative to expose the extent to which classical approaches to evaluation are valid and where new issues arise. Some problems highlighted include the difficulty of establishing a relevant baseline, the inability of public statistics constructed in supply-side mode to capture actions, the need to engage with actors who do not necessarily see themselves as part of the initiative being evaluated, long timescales and potential wide geographical scope, measures that span from micro to macro, and blurred boundaries between implementation and impact. It is concluded that there is a key role for evaluators to become involved in co-learning and co-evolution of these policy instruments in a manner analogous to the relationship between evaluation and policy development that characterized the emergence of collaborative R&D support programmes. Copyright The Author 2012. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oup.com, Oxford University Press.

Suggested Citation

  • J. Edler & L. Georghiou & K. Blind & E. Uyarra, 2012. "Evaluating the demand side: New challenges for evaluation," Research Evaluation, Oxford University Press, vol. 21(1), pages 33-47, February.
  • Handle: RePEc:oup:rseval:v:21:y:2012:i:1:p:33-47
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/reseval/rvr002
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Daniele Rotolo & Michael Hopkins & Nicola Grassano, 2023. "Do funding sources complement or substitute? Examining the impact of cancer research publications," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 74(1), pages 50-66, January.
    2. Ghisetti, Claudia, 2017. "Demand-pull and environmental innovations: Estimating the effects of innovative public procurement," Technological Forecasting and Social Change, Elsevier, vol. 125(C), pages 178-187.
    3. Falck, Oliver & Wiederhold, Simon, 2013. "Nachfrageorientierte Innovationspolitik," Studien zum deutschen Innovationssystem 12-2013, Expertenkommission Forschung und Innovation (EFI) - Commission of Experts for Research and Innovation, Berlin.
    4. Katarzyna Szkuta & Blagoy Stamenov & Anna Ianshyna, 2017. "Improving access to finance for young innovative enterprises with growth potential: evidence of impact on firms' outputs: Part 1. Equity instruments: lessons learned from policy evaluations," JRC Research Reports JRC109163, Joint Research Centre (Seville site).
    5. Obwegeser, Nikolaus & Müller, Sune Dueholm, 2018. "Innovation and public procurement: Terminology, concepts, and applications," Technovation, Elsevier, vol. 74, pages 1-17.
    6. Brian Belcher & Janet Halliwell, 2021. "Conceptualizing the elements of research impact: towards semantic standards," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-6, December.
    7. Jakob Edler & Luke Georghiou & Elvira Uyarra & Jillian Yeow, 2015. "The meaning and limitations of public procurement for innovation: a supplier’s experience," Chapters, in: Charles Edquist & Nicholas S Vonortas & Jon M Zabala-Iturriagagoitia & Jakob Edler (ed.), Public Procurement for Innovation, chapter 2, pages 35-64, Edward Elgar Publishing.
    8. Juliana Subtil Lacerda & Jeroen C. J. M. Van den Bergh, 2014. "International Diffusion of Renewable Energy Innovations: Lessons from the Lead Markets for Wind Power in China, Germany and USA," Energies, MDPI, vol. 7(12), pages 1-28, December.
    9. Charles Edquist & Nicholas S. Vonortas & Jon Mikel Zabala-Iturriagagoitia, 2015. "Introduction," Chapters, in: Charles Edquist & Nicholas S Vonortas & Jon M Zabala-Iturriagagoitia & Jakob Edler (ed.), Public Procurement for Innovation, chapter 1, pages 1-32, Edward Elgar Publishing.
    10. Oliver Falck & Simon Wiederhold, 2013. "Demand-Oriented Innovation Policy: A Critical Review," ifo Beiträge zur Wirtschaftsforschung, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, number 51.
    11. TAMURA Suguru & IWAMI Shino & SAKATA Ichiro, 2016. "Does Standardization Affect Science Linkage? Implications for society and organizational management: Evidence from artificial intelligence-related technology," Policy Discussion Papers 16007, Research Institute of Economy, Trade and Industry (RIETI).
    12. Greco, Marco & Germani, Francesca & Grimaldi, Michele & Radicic, Dragana, 2022. "Policy mix or policy mess? Effects of cross-instrumental policy mix on eco-innovation in German firms," Technovation, Elsevier, vol. 117(C).
    13. Congli Su & Mingxi Wang, 2022. "Quality incentive contract design in government procurement for innovation," Managerial and Decision Economics, John Wiley & Sons, Ltd., vol. 43(8), pages 3665-3684, December.
    14. Chicot, J. & Matt, M., 2015. "Public procurement of innovation: a review of rationales, instruments and design," Working Papers 2015-05, Grenoble Applied Economics Laboratory (GAEL).
    15. Reed, M.S. & Ferré, M. & Martin-Ortega, J. & Blanche, R. & Lawford-Rolfe, R. & Dallimer, M. & Holden, J., 2021. "Evaluating impact from research: A methodological framework," Research Policy, Elsevier, vol. 50(4).
    16. Jillian Yeow & Elvira Uyarra & Sally Gee, 2015. "Closing the loop: examining the case of the procurement of a sustainable innovation," Chapters, in: Charles Edquist & Nicholas S Vonortas & Jon M Zabala-Iturriagagoitia & Jakob Edler (ed.), Public Procurement for Innovation, chapter 9, pages 235-262, Edward Elgar Publishing.
    17. Martin Januska & Alena Palacka, 2023. "Critical success factors for public procurement of innovative solutions in Central Europe: Empirical study," E&M Economics and Management, Technical University of Liberec, Faculty of Economics, vol. 26(2), pages 24-41, June.
    18. Robert Gampfer & Jessica Mitchell & Blagoy Stamenow & Jana Zifciakova & Koen Jonkers, 2016. "Improving access to finance: which schemes best support the emergence of high-growth innovative enterprises? A mapping, analysis and assessment of finance instruments in selected EU Member States," JRC Research Reports JRC102928, Joint Research Centre (Seville site).
    19. Julien Chicot & Mireille Matt, 2018. "Public procurement of innovation: a review of rationales, designs, and contributions to grand challenges," Science and Public Policy, Oxford University Press, vol. 45(4), pages 480-492.
    20. Tamura, Suguru, 2016. "A new intellectual property metric for standardization activities," Technovation, Elsevier, vol. 48, pages 87-98.
    21. Adriana Bin & Sergio Salles-Filho & Luiza Maria Capanema & Fernando Antonio Basile Colugnati, 2015. "What difference does it make? Impact of peer-reviewed scholarships on scientific production," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1167-1188, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:rseval:v:21:y:2012:i:1:p:33-47. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/rev .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.