IDEAS home Printed from https://ideas.repec.org/p/ira/wpaper/202005.html
   My bibliography  Save this paper

Ensuring Stability, Accuracy and Meaningfulness in Synthetic Control Methods: The Regularized SHAP-Distance Method

Author

Listed:
  • Daniel Albalate

    (Department of Econometrics, Statistics and Applied Economics (Public Policy Unit). Universitat de Barcelona. John Keynes 1-11, 08034 Barcelona. Spain)

  • Germà Bel

    (Department of Econometrics, Statistics and Applied Economics (Public Policy Unit). Universitat de Barcelona. John Keynes 1-11, 08034 Barcelona. Spain)

  • Ferran A. Mazaira-Font

    (Department of Econometrics, Statistics and Applied Economics (Public Policy Unit). Universitat de Barcelona. John Keynes 1-11, 08034 Barcelona. Spain)

Abstract

The synthetic control method (SCM) has been increasingly adopted to evaluate causal effects under quasi-experimental designs. However, SCM suffers from sound weaknesses that compromise its accuracy, stability and meaningfulness. The SHAP-distance synthetic control method (SD-SCM) is proposed as solution. We evaluate the economic effect of the government formation deadlock in Spain, 2016. The deadlock did not negatively affect economic growth, as the economy grew 1.59% more without full government; standard SCM method overestimates the effect by 0.23 pp. We show that SD-SCM offers higher accuracy and stability, while ensuring the economic meaningfulness of covariates used in building the counterfactual.

Suggested Citation

  • Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2020. "Ensuring Stability, Accuracy and Meaningfulness in Synthetic Control Methods: The Regularized SHAP-Distance Method," IREA Working Papers 202005, University of Barcelona, Research Institute of Applied Economics, revised Apr 2020.
    • Handle: RePEc:ira:wpaper:202005
    as

    Download full text from publisher

    File URL: http://www.ub.edu/irea/working_papers/2020/202005.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Laurent Gobillon & Thierry Magnac, 2016. "Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 535-551, July.
    2. Vetlov, Igor & Ferdinandusse, Marien & de Jong, Jasper & Funda, Josip, 2017. "The effect of public investment in Europe: a model-based assessment," Working Paper Series 2021, European Central Bank.
    3. José G. Montalvo, 2011. "Voting after the Bombings: A Natural Experiment on the Effect of Terrorist Attacks on Democratic Elections," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1146-1154, November.
    4. Konstantinos Angelopoulos & George Economides, 2008. "Fiscal policy, rent seeking, and growth under electoral uncertainty: theory and evidence from the OECD," Canadian Journal of Economics, Canadian Economics Association, vol. 41(4), pages 1375-1405, November.
    5. Sarah Bohn & Magnus Lofstrom & Steven Raphael, 2014. "Did the 2007 Legal Arizona Workers Act Reduce the State's Unauthorized Immigrant Population?," The Review of Economics and Statistics, MIT Press, vol. 96(2), pages 258-269, May.
    6. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    7. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    8. Nikolay Doudchenko & Guido W. Imbens, 2016. "Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis," NBER Working Papers 22791, National Bureau of Economic Research, Inc.
    9. Grier, Kevin & Maynard, Norman, 2016. "The economic consequences of Hugo Chavez: A synthetic control analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 125(C), pages 1-21.
    10. Andreas Billmeier & Tommaso Nannicini, 2013. "Assessing Economic Liberalization Episodes: A Synthetic Control Approach," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 983-1001, July.
    11. Ozlem Akin & José Montalvo & Jaume García Villar & José-Luis Peydró & Josep Raya, 2014. "The real estate and credit bubble: evidence from Spain," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 5(2), pages 223-243, August.
    12. Noémi Kreif & Richard Grieve & Dominik Hangartner & Alex James Turner & Silviya Nikolova & Matt Sutton, 2016. "Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units," Health Economics, John Wiley & Sons, Ltd., vol. 25(12), pages 1514-1528, December.
    13. Aisen, Ari & Veiga, Francisco José, 2013. "How does political instability affect economic growth?," European Journal of Political Economy, Elsevier, vol. 29(C), pages 151-167.
    14. Dana E Goin & Kara E Rudolph & Jennifer Ahern, 2017. "Impact of drought on crime in California: A synthetic control approach," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-15, October.
    15. Marco Percoco, 2015. "Heterogeneity in the reaction of traffic flows to road pricing: a synthetic control approach applied to Milan," Transportation, Springer, vol. 42(6), pages 1063-1079, November.
    16. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2011. "Synth: An R Package for Synthetic Control Methods in Comparative Case Studies," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i13).
    17. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    18. Kaul, Ashok & Klößner, Stefan & Pfeifer, Gregor & Schieler, Manuel, 2015. "Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates," MPRA Paper 83790, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2021. "Decoupling synthetic control methods to ensure stability, accuracy and meaningfulness," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 12(4), pages 549-584, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2021. "Decoupling synthetic control methods to ensure stability, accuracy and meaningfulness," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 12(4), pages 549-584, December.
    2. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    3. Kaul, Ashok & Klößner, Stefan & Pfeifer, Gregor & Schieler, Manuel, 2015. "Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates," MPRA Paper 83790, University Library of Munich, Germany.
    4. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    5. Ferman, Bruno & Pinto, Cristine, 2016. "Revisiting the Synthetic Control Estimator," MPRA Paper 73982, University Library of Munich, Germany.
    6. Irene Botosaru & Bruno Ferman, 2019. "On the role of covariates in the synthetic control method," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 117-130.
    7. Giulio Grossi & Marco Mariani & Alessandra Mattei & Patrizia Lattarulo & Ozge Oner, 2020. "Direct and spillover effects of a new tramway line on the commercial vitality of peripheral streets. A synthetic-control approach," Papers 2004.05027, arXiv.org, revised Nov 2023.
    8. Pier Basaglia & Sophie Behr & Moritz A. Drupp, 2023. "De-Fueling Externalities: How Tax Salience and Fuel Substitution Mediate Climate and Health Benefits," Discussion Papers of DIW Berlin 2041, DIW Berlin, German Institute for Economic Research.
    9. Yi‐Ting Chen, 2020. "A distributional synthetic control method for policy evaluation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 35(5), pages 505-525, August.
    10. Pier Basaglia & Sophie M. Behr & Moritz A. Drupp, 2023. "De-Fueling Externalities: Causal Effects of Fuel Taxation and Mediating Mechanisms for Reducing Climate and Pollution Costs," CESifo Working Paper Series 10508, CESifo.
    11. Cruz A. Echevarría & Javier García-Enríquez, 2020. "The economic cost of the Arab Spring: the case of the Egyptian revolution," Empirical Economics, Springer, vol. 59(3), pages 1453-1477, September.
    12. Samuel Verevis & Murat Üngör, 2021. "What has New Zealand gained from The FTA with China?: Two counterfactual analyses†," Scottish Journal of Political Economy, Scottish Economic Society, vol. 68(1), pages 20-50, February.
    13. Kuosmanen, Timo & Zhou, Xun & Eskelinen, Juha & Malo, Pekka, 2021. "Design Flaw of the Synthetic Control Method," MPRA Paper 106328, University Library of Munich, Germany.
    14. Maïmouna Diakite & Jean-François Brun & Souleymane Diarra & Nasser Ary Tanimoune, 2017. "The effects of tax coordination on the tax revenue mobilization in West African Economic and Monetary Union (WAEMU)," CERDI Working papers halshs-01535104, HAL.
    15. Becker, Maike & Pfeifer, Gregor & Schweikert, Karsten, 2021. "Price Effects of the Austrian Fuel Price Fixing Act: A Synthetic Control Study," Energy Economics, Elsevier, vol. 97(C).
    16. Klößner, Stefan & Pfeifer, Gregor, 2015. "Synthesizing Cash for Clunkers: Stabilizing the Car Market, Hurting the Environment," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113207, Verein für Socialpolitik / German Economic Association.
    17. Kleis, Mischa & Moessinger, Marc-Daniel, 2016. "The long-run effect of fiscal consolidation on economic growth: Evidence from quantitative case studies," ZEW Discussion Papers 16-047, ZEW - Leibniz Centre for European Economic Research, revised 2016.
    18. Matej Opatrny, 2021. "The impact of the Brexit vote on UK financial markets: a synthetic control method approach," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 48(2), pages 559-587, May.
    19. Emery, Thomas & Mélon, Lela & Spruk, Rok, 2023. "Does e-procurement matter for economic growth? Subnational evidence from Australia," The Quarterly Review of Economics and Finance, Elsevier, vol. 89(C), pages 318-334.
    20. Kerianne Lawson, 2023. "Using property rights to fight crime: the Khaya Lam project," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 47(2), pages 269-302, June.

    More about this item

    Keywords

    Synthetic control; SHAP; Regularization; Quasiexperiments; Causality; Government. JEL classification: C32; E65; H11.;
    All these keywords.

    JEL classification:

    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • E65 - Macroeconomics and Monetary Economics - - Macroeconomic Policy, Macroeconomic Aspects of Public Finance, and General Outlook - - - Studies of Particular Policy Episodes

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ira:wpaper:202005. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Alicia García (email available below). General contact details of provider: https://edirc.repec.org/data/feubaes.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.