Evaluating the impact of public policies on large firms: a synthetic control approach to science industry transfer policies

Large firms dominate R&D investment in most countries and receive the majority of public R&D funding. Due to methodological difficulties, however, evaluation of the effect of government-sponsored R&D programmes mainly focuses on small-and medium-sized enterprises. The scarcity of large firms and their heterogeneity hampers the ability to find proper counterfactuals for very large companies and makes it difficult to use proper inference methods to measure the impact of a specific policy. In order to address these methodological issues, we propose using the synthetic control method, initially developed by Abadie et al. (2010) to evaluate programmes on a regional scale. We apply this method to evaluate the impact of a new French science-industry transfer initiative and compare the results with the random trend model and more standard counterfactual approaches. Based on data covering a long pre-treatment period (1998-2011) and ongoing treatment period (2012-2015), we reveal a convergence between the results obtained with the synthetic control method and the random trend model, and demonstrate that traditional counterfactual evaluation methods are not appropriate for large firms. Moreover, the synthetic control method has the advantage of providing an individual assessment of the policy impact on each firm. In the specific case of the French science-industry transfer initiative, it reveals that the impact on private R&D is highly heterogenous both on RD inputs and cooperation behaviours. Beyond this specific transfer policy, this study suggests that the synthetic control method opens new research perspectives in policy impact evaluation at the firm level. Abstract: Large firms dominate R&D investment in most countries and receive the majority of public R&D funding. Due to methodological difficulties, however, evaluation of the effect of government-sponsored R&D programmes mainly focuses on small-and medium-sized enterprises. The scarcity of large firms and their heterogeneity hampers the ability to find proper counterfactuals for very large companies and makes it difficult to use proper inference methods to measure the impact of a specific policy. In order to address these methodological issues, we propose using the synthetic control method, initially developed by Abadie et al. (2010) to evaluate programmes on a regional scale. We apply this method to evaluate the impact of a new French science-industry transfer initiative and compare the results with the random trend model and more standard counterfactual approaches. Based on data covering a long pre-treatment period (1998-2011) and ongoing treatment period (2012-2015), we reveal a convergence between the results obtained with the synthetic control method and the random trend model, and demonstrate that traditional counterfactual evaluation methods are not appropriate for large firms. Moreover, the synthetic control method has the advantage of providing an individual assessment of the policy impact on each firm. In the specific case of the French science-industry transfer initiative, it reveals that the impact on private R&D is highly heterogenous both on RD inputs and cooperation behaviours. Beyond this specific transfer policy, this study suggests that the synthetic control method opens new research perspectives in policy impact evaluation at the firm level.