IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v104y2024ics0149718924000314.html

Beyond the average effect of the innovation subsidies: Using case selection via matching to break impasse in delivering useful advice to policy makers

Author

Listed:
  • Koniewski, Maciej
  • Krupnik, Seweryn
  • Skórska, Paulina

Abstract

Experts and stakeholders promote the combined use of counterfactual and theory-based approaches in program evaluation. We illustrated combined application of these two approaches in a single evaluation study of innovation subsidies, using “case selection via matching” and follow-up in-depth interviews. We conducted interviews in contrasting pairs of companies—one successful and one unsuccessful—which were otherwise similar on a defined set of covariates. Our procedure helped to reveal factors, which might be overlooked or simply not available in data at hand and hence not accounted for in analyses of the intervention effects. As such it extends beyond the average effect estimate to highlight causes of an intervention success or failure.

Suggested Citation

  • Koniewski, Maciej & Krupnik, Seweryn & Skórska, Paulina, 2024. "Beyond the average effect of the innovation subsidies: Using case selection via matching to break impasse in delivering useful advice to policy makers," Evaluation and Program Planning, Elsevier, vol. 104(C).
    • Handle: RePEc:eee:epplan:v:104:y:2024:i:c:s0149718924000314
    DOI: 10.1016/j.evalprogplan.2024.102429
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718924000314
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2024.102429?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to

    for a different version of it.

    References listed on IDEAS

    as
    1. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates," American Economic Review, American Economic Association, vol. 104(9), pages 2593-2632, September.
    2. Howard White, 2009. "Theory-based impact evaluation: principles and practice," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 1(3), pages 271-284.
    3. David Bartle & Michele Morris, 2010. "Evaluating the impacts of government business assistance programmes: approaches to testing additionality," Research Evaluation, Oxford University Press, vol. 19(4), pages 275-280, October.
    4. Lijphart, Arend, 1971. "Comparative Politics and the Comparative Method," American Political Science Review, Cambridge University Press, vol. 65(3), pages 682-693, September.
    5. Nanny Bressers & Lasse Gerrits, 2015. "A Complexity-Informed Approach to Evaluating National Knowledge and Innovation Programmes," Systems Research and Behavioral Science, Wiley Blackwell, vol. 32(1), pages 50-63, January.
    6. Kane, Thomas J. & Rockoff, Jonah E. & Staiger, Douglas O., 2008. "What does certification tell us about teacher effectiveness? Evidence from New York City," Economics of Education Review, Elsevier, vol. 27(6), pages 615-631, December.
    7. James J. Heckman, 1989. "Causal Inference and Nonrandom Samples," Journal of Educational and Behavioral Statistics, , vol. 14(2), pages 159-168, June.
    8. José Ángel Zúñiga-Vicente & César Alonso-Borrego & Francisco J. Forcadell & José I. Galán, 2014. "Assessing The Effect Of Public Subsidies On Firm R&D Investment: A Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 28(1), pages 36-67, February.
    9. Falck, Oliver & Koenen, Johannes & Lohse, Tobias, 2019. "Evaluating a place-based innovation policy: Evidence from the innovative Regional Growth Cores Program in East Germany," Regional Science and Urban Economics, Elsevier, vol. 79(C).
    10. Lieberman, Evan S., 2005. "Nested Analysis as a Mixed-Method Strategy for Comparative Research," American Political Science Review, Cambridge University Press, vol. 99(3), pages 435-452, August.
    11. Chen, Huey-Tsyh & Rossi, Peter H., 1989. "Issues in the theory-driven perspective," Evaluation and Program Planning, Elsevier, vol. 12(4), pages 299-306, January.
    12. Iacus, Stefano M. & King, Gary & Porro, Giuseppe, 2012. "Causal Inference without Balance Checking: Coarsened Exact Matching," Political Analysis, Cambridge University Press, vol. 20(1), pages 1-24, January.
    13. Michele Morris & Oliver J. Herrmann, 2013. "Beyond surveys: The research frontier moves to the use of administrative data to evaluate R&D grants," Research Evaluation, Oxford University Press, vol. 22(5), pages 298-306, October.
    14. Dominique Guellec & Bruno Van Pottelsberghe De La Potterie, 2003. "The impact of public R&D expenditure on business R&D," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 12(3), pages 225-243.
    15. James Heckman, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    16. Bracic, Ana, 2016. "Reaching the Individual: EU Accession, NGOs, and Human Rights," American Political Science Review, Cambridge University Press, vol. 110(3), pages 530-546, August.
    17. Madrigal, Róger & Alpízar, Francisco & Schlüter, Achim, 2011. "Determinants of Performance of Community-Based Drinking Water Organizations," World Development, Elsevier, vol. 39(9), pages 1663-1675, September.
    18. King, Gary & Nielsen, Richard, 2019. "Why Propensity Scores Should Not Be Used for Matching," Political Analysis, Cambridge University Press, vol. 27(4), pages 435-454, October.
    19. Marino, Marianna & Lhuillery, Stephane & Parrotta, Pierpaolo & Sala, Davide, 2016. "Additionality or crowding-out? An overall evaluation of public R&D subsidy on private R&D expenditure," Research Policy, Elsevier, vol. 45(9), pages 1715-1730.
    20. Colin Elman & John Gerring & James Mahoney, 2016. "Case Study Research," Sociological Methods & Research, , vol. 45(3), pages 375-391, August.
    21. Giovanni Cerulli & Bianca Potì, 2012. "Evaluating the Robustness of the Effect of Public Subsidies on Firms' R&D: An Application to Italy," Journal of Applied Economics, Taylor & Francis Journals, vol. 15(2), pages 287-320, November.
    22. Spyros Arvanitis, 2013. "Micro-econometric approaches to the evaluation of technology-oriented public programmes: a non-technical review of the state of the art," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 3, pages 56-88, Edward Elgar Publishing.
    23. Wanzenböck, Iris & Scherngell, Thomas & Fischer, Manfred M., 2013. "How do firm characteristics affect behavioural additionalities of public R&D subsidies? Evidence for the Austrian transport sector," MPRA Paper 77552, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ugur, Mehmet & Trushin, Eshref, 2018. "Asymmetric information and heterogeneous effects of R&D subsidies: evidence on R&D investment and employment of R&D personel," Greenwich Papers in Political Economy 21943, University of Greenwich, Greenwich Political Economy Research Centre.
    2. Bleda, Mercedes & Krupnik, Seweryn, 2024. "Risks of policy failure in direct R&D support," Technological Forecasting and Social Change, Elsevier, vol. 209(C).
    3. Michel Dumont, 2019. "Working Paper 04-19 - Tax incentives for business R&D in Belgium - Third evaluation," Working Papers 1904, Federal Planning Bureau, Belgium.
    4. Dumont, Michel, 2017. "Assessing the policy mix of public support to business R&D," Research Policy, Elsevier, vol. 46(10), pages 1851-1862.
    5. Huergo, Elena & Trenado, Mayte & Ubierna, Andrés, 2016. "The impact of public support on firm propensity to engage in R&D: Spanish experience," Technological Forecasting and Social Change, Elsevier, vol. 113(PB), pages 206-219.
    6. Thomas H. W. Ziesemer, 2021. "The Effects of R&D Subsidies and Publicly Performed R&D on Business R&D: A Survey," Hacienda Pública Española / Review of Public Economics, IEF, vol. 236(1), pages 171-205, March.
    7. Dong Xiang & Roman Matousek & Andrew C. Worthington & Yue Jiang, 2025. "How Do Government Subsidies Affect Innovation? Evidence from Chinese Hi-Tech SMEs," Sustainability, MDPI, vol. 17(15), pages 1-23, August.
    8. Daniel Blaseg & Armin Schwienbacher, 2024. "Biased Calibration: Exacerbating Instead of Mitigating Entrepreneurial Overplacement with Reference Values," Entrepreneurship Theory and Practice, , vol. 48(5), pages 1131-1159, September.
    9. Lee, Jeongwon & Hwang, Junseok & Kim, Hana, 2022. "Different government support effects on emerging and mature ICT sectors," Technological Forecasting and Social Change, Elsevier, vol. 174(C).
    10. Eliane Badaoui & Eleonora Matteazzi & Vincenzo Prete, 2024. "The role of non‐base compensation in explaining the motherhood wage gap: Evidence from Italy," Kyklos, Wiley Blackwell, vol. 77(4), pages 873-894, November.
    11. Beck, Mathias & Junge, Martin & Kaiser, Ulrich, 2017. "Public Funding and Corporate Innovation," IZA Discussion Papers 11196, IZA Network @ LISER.
    12. Archibugi, Daniele & Filippetti, Andrea, 2018. "The retreat of public research and its adverse consequences on innovation," Technological Forecasting and Social Change, Elsevier, vol. 127(C), pages 97-111.
    13. Maarten Goos & Anna Salomons, 2017. "Measuring teaching quality in higher education: assessing selection bias in course evaluations," Research in Higher Education, Springer;Association for Institutional Research, vol. 58(4), pages 341-364, June.
    14. Cristiano Antonelli, 2020. "Knowledge exhaustibility public support to business R&D and the additionality constraint," The Journal of Technology Transfer, Springer, vol. 45(3), pages 649-663, June.
    15. Carolina Caetano & Gregorio Caetano & Eric R. Nielsen, 2020. "Should Children Do More Enrichment Activities? Leveraging Bunching to Correct for Endogeneity," Finance and Economics Discussion Series 2020-036, Board of Governors of the Federal Reserve System (U.S.).
    16. Yidi Liu & Xin Li & Zhiqiang (Eric) Zheng, 2024. "Smart Natural Disaster Relief: Assisting Victims with Artificial Intelligence in Lending," Information Systems Research, INFORMS, vol. 35(2), pages 489-504, June.
    17. Szücs, Florian, 2020. "Do research subsidies crowd out private R&D of large firms? Evidence from European Framework Programmes," Research Policy, Elsevier, vol. 49(3).
    18. Wanshu Wu & Kai Zhao & Lei Li, 2021. "Can government subsidy strategies and strategy combinations effectively stimulate enterprise innovation? Theory and evidence," Economia Politica: Journal of Analytical and Institutional Economics, Springer;Fondazione Edison, vol. 38(2), pages 423-446, July.
    19. Dai, Xiaoyong & Cheng, Liwei, 2015. "The effect of public subsidies on corporate R&D investment: An application of the generalized propensity score," Technological Forecasting and Social Change, Elsevier, vol. 90(PB), pages 410-419.
    20. Blázquez, Maite & Herrarte, Ainhoa & Moro-Egido, Ana I., 2024. "Has the COVID-19 pandemic widened the gender gap in paid work hours in Spain?," Journal of Demographic Economics, Cambridge University Press, vol. 90(2), pages 313-348, June.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:104:y:2024:i:c:s0149718924000314. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.