IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v73y2019icp146-155.html
   My bibliography  Save this article

Using mixed method approach in measuring effects of training in firms: Case study of the European Social Fund support

Author

Listed:
  • Pelucha, Martin
  • Kveton, Viktor
  • Potluka, Oto

Abstract

Public support of training in firms corresponds to the long-term importance of the quality of human capital in the competitiveness of firms and nations. Thus, the EU supports such training via the European Social Fund (ESF). The evaluation community evaluates the support by using either qualitative or quantitative methods. The simultaneous application of these two approaches is rare. The purpose of this paper is to combine quantitative (counterfactual impact evaluation) and qualitative (qualitative comparative analysis) methods in order to fill the methodological gap. Based on the combination of both approaches, it explores their strengths, complementarity and disadvantages to evaluate public support for employee training in the Czech Republic. The combination of methods makes it possible to identify not only the impacts but also their causes. Linking the ESF support to corporate competitiveness is crucial for demonstrating the effectiveness of public spending.

Suggested Citation

  • Pelucha, Martin & Kveton, Viktor & Potluka, Oto, 2019. "Using mixed method approach in measuring effects of training in firms: Case study of the European Social Fund support," Evaluation and Program Planning, Elsevier, vol. 73(C), pages 146-155.
    • Handle: RePEc:eee:epplan:v:73:y:2019:i:c:p:146-155
    DOI: 10.1016/j.evalprogplan.2018.12.008
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718918300843
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2018.12.008?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alan Barrett & Philip J. O'Connell, 2001. "Does Training Generally Work? The Returns to in-Company Training," ILR Review, Cornell University, ILR School, vol. 54(3), pages 647-662, April.
    2. Andries De Grip & Jan Sauermann, 2012. "The Effects of Training on Own and Co‐worker Productivity: Evidence from a Field Experiment," Economic Journal, Royal Economic Society, vol. 122(560), pages 376-399, May.
    3. Aguado Loi, Claudia X. & Alfonso, Moya L. & Chan, Isabella & Anderson, Kelsey & Tyson, Dinorah (Dina) Martinez & Gonzales, Junius & Corvin, Jaime, 2017. "Application of mixed-methods design in community-engaged research: Lessons learned from an evidence-based intervention for Latinos with chronic illness and minor depression," Evaluation and Program Planning, Elsevier, vol. 63(C), pages 29-38.
    4. Frank Vanclay, 2015. "The Potential Application of Qualitative Evaluation Methods in European Regional Development: Reflections on the Use of Performance Story Reporting in Australian Natural Resource Management," Regional Studies, Taylor & Francis Journals, vol. 49(8), pages 1326-1339, August.
    5. Alecke Björn & Reinkowski Janina & Mitze Timo & Untiedt Gerhard, 2012. "Does Firm Size make a Difference? Analysing the Effectiveness of R&D Subsidies in East Germany," German Economic Review, De Gruyter, vol. 13(2), pages 174-195, May.
    6. Czarnitzki, Dirk & Lopes Bento, Cindy & Doherr, Thorsten, 2011. "Counterfactual impact evaluation of cohesion policy. Work package 2: Examples from support to innovation and research. Final report," ZEW Expertises, ZEW - Leibniz Centre for European Economic Research, number 110541.
    7. Betzner, Anne & Lawrenz, Frances P. & Thao, Mao, 2016. "Examining mixing methods in an evaluation of a smoking cessation program," Evaluation and Program Planning, Elsevier, vol. 54(C), pages 94-101.
    8. Girma, Sourafel & Görg, Holger & Strobl, Eric & Walsh, Frank, 2008. "Creating jobs through public subsidies: An empirical analysis," Labour Economics, Elsevier, vol. 15(6), pages 1179-1199, December.
    9. Bamberger, Michael & Tarsilla, Michele & Hesse-Biber, Sharlene, 2016. "Why so many “rigorous” evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute," Evaluation and Program Planning, Elsevier, vol. 55(C), pages 155-162.
    10. Harley Frazis & Maury Gittleman & Mary Joyce, 2000. "Correlates of Training: An Analysis Using Both Employer and Employee Characteristics," ILR Review, Cornell University, ILR School, vol. 53(3), pages 443-462, April.
    11. Makrakis, Vassilios & Kostoulas-Makrakis, Nelly, 2016. "Bridging the qualitative–quantitative divide: Experiences from conducting a mixed methods evaluation in the RUCAS programme," Evaluation and Program Planning, Elsevier, vol. 54(C), pages 144-151.
    12. James Foreman-Peck, 2013. "Effectiveness and efficiency of SME innovation policy," Small Business Economics, Springer, vol. 41(1), pages 55-70, June.
    13. Guido Pellegrini & Teo Muccigrosso, 2017. "Do subsidized new firms survive longer? Evidence from a counterfactual approach," Regional Studies, Taylor & Francis Journals, vol. 51(10), pages 1483-1493, October.
    14. Phillip Toner, 2011. "Workforce Skills and Innovation: An Overview of Major Themes in the Literature," OECD Science, Technology and Industry Working Papers 2011/1, OECD Publishing.
    15. Frondel, Manuel & Schmidt, Christoph M., 2005. "Evaluating environmental programs: The perspective of modern evaluation research," Ecological Economics, Elsevier, vol. 55(4), pages 515-526, December.
    16. Krueger, Alan & Rouse, Cecilia, 1998. "The Effect of Workplace Education on Earnings, Turnover, and Job Performance," Journal of Labor Economics, University of Chicago Press, vol. 16(1), pages 61-94, January.
    17. Walton, Mat, 2014. "Applying complexity theory: A review to inform evaluation design," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 119-126.
    18. Sekhobo, Jackson P. & Peck, Sanya R. & Byun, Youjung & Allsopp, Marie A.K. & Holbrook, MaryEllen K. & Edmunds, Lynn S. & Yu, Chengxuan, 2017. "Use of a mixed-method approach to evaluate the implementation of retention promotion strategies in the New York State WIC program," Evaluation and Program Planning, Elsevier, vol. 63(C), pages 7-17.
    19. McGuirk, Helen & Lenihan, Helena & Hart, Mark, 2015. "Measuring the impact of innovative human capital on small firms’ propensity to innovate," Research Policy, Elsevier, vol. 44(4), pages 965-976.
    20. Meschede, Tatjana & Chaganti, Sara, 2015. "Home for now: A mixed-methods evaluation of a short-term housing support program for homeless families," Evaluation and Program Planning, Elsevier, vol. 52(C), pages 85-95.
    21. Sarah Hamersma, 2008. "The effects of an employer subsidy on employment outcomes: A study of the work opportunity and welfare-to-work tax credits," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 27(3), pages 498-520.
    22. Elias Einiö, 2014. "R&D Subsidies and Company Performance: Evidence from Geographic Variation in Government Funding Based on the ERDF Population-Density Rule," The Review of Economics and Statistics, MIT Press, vol. 96(4), pages 710-728, October.
    23. Bloch, Carter & Sørensen, Mads P. & Graversen, Ebbe K. & Schneider, Jesper W. & Schmidt, Evanthia Kalpazidou & Aagaard, Kaare & Mejlgaard, Niels, 2014. "Developing a methodology to assess the impact of research grant funding: A mixed methods approach," Evaluation and Program Planning, Elsevier, vol. 43(C), pages 105-117.
    24. Betcherman, Gordon & Daysal, N. Meltem & Pagés, Carmen, 2010. "Do employment subsidies work? Evidence from regionally targeted subsidies in Turkey," Labour Economics, Elsevier, vol. 17(4), pages 710-722, August.
    25. Laura Abramovsky & Erich Battistin & Emla Fitzsimons & Alissa Goodman & Helen Simpson, 2011. "Providing Employers with Incentives to Train Low-SkilledWorkers: Evidence from the UK Employer Training Pilots," Journal of Labor Economics, University of Chicago Press, vol. 29(1), pages 153-193, January.
    26. Phillip Toner, 2011. "Workforce Skills and Innovation: An Overview of Major Themes in the Literature," OECD Education Working Papers 55, OECD Publishing.
    27. Ondřej Dvouletý, 2017. "Effects of Soft Loans and Credit Guarantees on Performance of Supported Firms: Evidence from the Czech Public Programme START," Sustainability, MDPI, vol. 9(12), pages 1-17, December.
    28. Celik, Halime & Abma, Tineke A. & Klinge, Ineke & Widdershoven, Guy A.M., 2012. "Process evaluation of a diversity training program: The value of a mixed method strategy," Evaluation and Program Planning, Elsevier, vol. 35(1), pages 54-65.
    29. Arco-Tirado, J.L. & Fernández-Martín, F. & Ramos-García, A.M. & Littvay, L. & Villoria, J. & Naranjo, J.A., 2018. "A counterfactual impact evaluation of a bilingual program on students’ grade point average at a spanish university," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 81-89.
    30. repec:ecj:econjl:v:122:y:2012:i::p:376-399 is not listed on IDEAS
    31. Inge Sieben, 2007. "Does training trigger turnover - or not?," Work, Employment & Society, British Sociological Association, vol. 21(3), pages 397-416, September.
    32. Davide Antonioli & Alberto Marzucchi & Sandro Montresor, 2014. "Regional Innovation Policy and Innovative Behaviour: Looking for Additional Effects," European Planning Studies, Taylor & Francis Journals, vol. 22(1), pages 64-83, January.
    33. Juan L. Martinez-Covarrubias & Helena Lenihan & Mark Hart, 2017. "Public support for business innovation in Mexico: a cross-sectional analysis," Regional Studies, Taylor & Francis Journals, vol. 51(12), pages 1786-1800, December.
    34. Jackson, Karen Moran & Pukys, Suzy & Castro, Andrene & Hermosura, Lorna & Mendez, Joanna & Vohra-Gupta, Shetal & Padilla, Yolanda & Morales, Gabriela, 2018. "Using the transformative paradigm to conduct a mixed methods needs assessment of a marginalized community: Methodological lessons and implications," Evaluation and Program Planning, Elsevier, vol. 66(C), pages 111-119.
    35. Hayes, Holly & Scott, Victoria & Abraczinskas, Michelle & Scaccia, Jonathan & Stout, Soma & Wandersman, Abraham, 2016. "A formative multi-method approach to evaluating training," Evaluation and Program Planning, Elsevier, vol. 58(C), pages 199-207.
    36. Hanushek, Eric A., 2013. "Economic growth in developing countries: The role of human capital," Economics of Education Review, Elsevier, vol. 37(C), pages 204-212.
    37. Harry J. Holzer & Richard N. Block & Marcus Cheatham & Jack H. Knott, 1993. "Are Training Subsidies for Firms Effective? The Michigan Experience," ILR Review, Cornell University, ILR School, vol. 46(4), pages 625-636, July.
    38. Stefaan Decramer & Stijn Vanormelingen, 2016. "The effectiveness of investment subsidies: evidence from a regression discontinuity design," Small Business Economics, Springer, vol. 47(4), pages 1007-1032, December.
    39. Sarah Batterbury, 2006. "Principles and purposes of European Union Cohesion policy evaluation," Regional Studies, Taylor & Francis Journals, vol. 40(2), pages 179-188.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Panagiotis Koudoumakis & George Botzoris & Angelos Protopapas, 2022. "Cohesion policy evaluation: Guidelines for selection of appropriate methods," Regional Science Policy & Practice, Wiley Blackwell, vol. 14(5), pages 1062-1084, October.
    2. Lucia Švábová & Katarína Kramárová & Marek Ďurica, 2021. "Evaluation of the Effects of the Graduate Practice in Slovakia: Comparison of Results of Counterfactual Methods," Central European Business Review, Prague University of Economics and Business, vol. 2021(4), pages 1-31.
    3. Wanda Biedka & Mikołaj Herbst & Jakub Rok & Piotr Wójcik, 2022. "The local‐level impact of human capital investment within the EU cohesion policy in Poland," Papers in Regional Science, Wiley Blackwell, vol. 101(2), pages 303-325, April.
    4. Alic BÎRCA, 2023. "Workforce Participation in Active Labour Market Policies: A Comparative Analysis in EU Member States," REVISTA DE MANAGEMENT COMPARAT INTERNATIONAL/REVIEW OF INTERNATIONAL COMPARATIVE MANAGEMENT, Faculty of Management, Academy of Economic Studies, Bucharest, Romania, vol. 24(2), pages 173-187, May.
    5. Plzakova, Lucie, 2022. "Evaluation of investments in the tourism sector with a local focus," Evaluation and Program Planning, Elsevier, vol. 94(C).
    6. Svabova, Lucia & Kramarova, Katarina, 2021. "An analysis of participation factors and effects of the active labour market measure Graduate practice in Slovakia – Counterfactual approach," Evaluation and Program Planning, Elsevier, vol. 86(C).
    7. Ferraro, Simona & Männasoo, Kadri & Tasane, Helery, 2023. "How the EU Cohesion Policy targeted at R&D and innovation impacts the productivity, employment and exports of SMEs in Estonia," Evaluation and Program Planning, Elsevier, vol. 97(C).
    8. Lucia Svabova & Eva Nahalkova Tesarova & Marek Durica & Lenka Strakova, 2021. "Evaluation of the impacts of the COVID-19 pandemic on the development of the unemployment rate in Slovakia: counterfactual before-after comparison," Equilibrium. Quarterly Journal of Economics and Economic Policy, Institute of Economic Research, vol. 16(2), pages 261-284, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dan A. Black & Lars Skipper & Jeffrey A. Smith & Jeffrey Andrew Smith, 2023. "Firm Training," CESifo Working Paper Series 10268, CESifo.
    2. Martins, Pedro S., 2021. "Employee training and firm performance: Evidence from ESF grant applications," Labour Economics, Elsevier, vol. 72(C).
    3. Pedro Martins, 2020. "Employee Training and Firm Performance:Quasi-experimental evidence from the European Social Fund," CVER Research Papers 027, Centre for Vocational Education Research.
    4. Ondřej Dvouletý & Stjepan Srhoj & Smaranda Pantea, 2021. "Public SME grants and firm performance in European Union: A systematic review of empirical evidence," Small Business Economics, Springer, vol. 57(1), pages 243-263, June.
    5. Kiman Kim & Sang Ok Choi & Sooyeon Lee, 2021. "The Effect of a Financial Support on Firm Innovation Collaboration and Output: Does Policy Work on the Diverse Nature of Firm Innovation?," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 12(2), pages 645-675, June.
    6. Valeria Cirillo & Lucrezia Fanti & Andrea Mina & Andrea Ricci, 2021. "Digitalizing Firms: Skills, Work Organization and the Adoption of New Enabling Technologies," LEM Papers Series 2021/04, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy.
    7. Caliendo, Marco & Cobb-Clark, Deborah A. & Pfeifer, Harald & Uhlendorff, Arne & Wehner, Caroline, 2024. "Managers’ risk preferences and firm training investments," European Economic Review, Elsevier, vol. 161(C).
    8. Medase, Kehinde, 2019. "The Impact of the Heterogeneity of Employees’ Qualifications on Firm-level Innovation Evidence from Nigerian Firms," VfS Annual Conference 2019 (Leipzig): 30 Years after the Fall of the Berlin Wall - Democracy and Market Economy 203563, Verein für Socialpolitik / German Economic Association.
    9. Le Quang Hieu & Vu Hong Van & Nguyen Khanh Doanh, 2023. "Impacts of switching from business households to enterprises on the financial performance," Portuguese Economic Journal, Springer;Instituto Superior de Economia e Gestao, vol. 22(3), pages 417-437, September.
    10. Addison, John T. & Belfield, Clive R., 2004. "Unions, Training, and Firm Performance: Evidence from the British Workplace Employee Relations Survey," IZA Discussion Papers 1264, Institute of Labor Economics (IZA).
    11. Laura-Diana Radu, 2012. "Technological Innovation Management and its Role in Performance of Organizations," Acta Universitatis Danubius. OEconomica, Danubius University of Galati, issue 5(5), pages 95-103, October.
    12. Horbach, Jens & Rammer, Christian, 2020. "Labor shortage and innovation," ZEW Discussion Papers 20-009, ZEW - Leibniz Centre for European Economic Research.
    13. Soriano, Franklin A. & Villano, Renato A. & Fleming, Euan M. & Battese, George E., 2018. "What’s driving innovation in small businesses in Australia? The case of the food industry," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 63(1), October.
    14. Dario Blanco-Fernandez & Stephan Leitner & Alexandra Rausch, 2022. "Interactions between the individual and the group level in organizations: The case of learning and autonomous group adaptation," Papers 2203.09162, arXiv.org.
    15. Dragana Radicic & Geoffrey Pugh & David Douglas, 2020. "Promoting cooperation in innovation ecosystems: evidence from European traditional manufacturing SMEs," Small Business Economics, Springer, vol. 54(1), pages 257-283, January.
    16. Dimos, Christos & Pugh, Geoff & Hisarciklilar, Mehtap & Talam, Ema & Jackson, Ian, 2022. "The relative effectiveness of R&D tax credits and R&D subsidies: A comparative meta-regression analysis," Technovation, Elsevier, vol. 115(C).
    17. Dragana Radicic & Geoffrey Pugh & Hugo Hollanders & René Wintjes & Jon Fairburn, 2016. "The impact of innovation support programs on small and medium enterprises innovation in traditional manufacturing industries: An evaluation for seven European Union regions," Environment and Planning C, , vol. 34(8), pages 1425-1452, December.
    18. Frank Barry & Holger Görg & Eric Strobl, 2004. "Multinationals and Training: Some Evidence from Irish Manufacturing Industries," Scottish Journal of Political Economy, Scottish Economic Society, vol. 51(1), pages 49-61, February.
    19. Floreani, Vincent Arthur, 2014. "Fixing Europe's youth unemployment and skills mismatch, can public financial support to SMEs be effective? The case of the European Commission and European Investment Bank joint initiatives," MPRA Paper 55849, University Library of Munich, Germany.
    20. Sungsup Ra & Brian Chin & Amy Liu, 2017. "Challenges and Opportunities for Skills Development in Asia: Changing Supply, Demand, and Mismatches," Working Papers id:11838, eSocialSciences.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:73:y:2019:i:c:p:146-155. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.