IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/73982.html
   My bibliography  Save this paper

Revisiting the Synthetic Control Estimator

Author

Listed:
  • Ferman, Bruno
  • Pinto, Cristine

Abstract

The synthetic control (SC) method has been recently proposed as an alternative to estimate treatment effects in comparative case studies. The SC relies on the assumption that there is a weighted average of the control units that reconstructs the potential outcome of the treated unit in the absence of treatment. If these weights were known, then constructing the counterfactual for the treated unit using a weighted average of the control units would provide an unbiased estimator for the treatment effect, even if selection into treatment is correlated with the unobserved heterogeneity. In this paper, we revisit the SC method in a linear factor model where the SC weights are considered nuisance parameters that are estimated to construct the SC estimator. We show that, when the number of control units is fixed, the estimated SC weights will generally not converge to the weights that reconstruct the factor loadings of the treated unit, even when the number of pre-intervention periods goes to infinity. As a consequence, the SC estimator will be asymptotically biased if treatment assignment is correlated with the unobserved heterogeneity. The asymptotic bias only vanishes when the variance of the idiosyncratic error goes to zero. We suggest a slight modification in the SC method that guarantees that the SC estimator is asymptotically unbiased and has a lower asymptotic variance than the difference-in-differences (DID) estimator when the DID identification assumption is satisfied. We also propose an alternative way to estimate the SC weights that provides an asymptotically unbiased estimator under additional assumptions on the error structure. Finally, we consider the implications of our findings to the permutation test suggested in Abadie et al. (2010).

Suggested Citation

  • Ferman, Bruno & Pinto, Cristine, 2016. "Revisiting the Synthetic Control Estimator," MPRA Paper 73982, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:73982
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/73982/1/MPRA_paper_73982.pdf
    File Function: original version
    Download Restriction: no

    File URL: https://mpra.ub.uni-muenchen.de/75128/1/MPRA_paper_75128.pdf
    File Function: revised version
    Download Restriction: no

    File URL: https://mpra.ub.uni-muenchen.de/77930/1/MPRA_paper_77930.pdf
    File Function: revised version
    Download Restriction: no

    File URL: https://mpra.ub.uni-muenchen.de/80685/1/MPRA_paper_80685.pdf
    File Function: revised version
    Download Restriction: no

    File URL: https://mpra.ub.uni-muenchen.de/81941/1/MPRA_paper_81941.pdf
    File Function: revised version
    Download Restriction: no

    File URL: https://mpra.ub.uni-muenchen.de/86495/1/MPRA_paper_86495.pdf
    File Function: revised version
    Download Restriction: no

    File URL: https://mpra.ub.uni-muenchen.de/95524/1/MPRA_paper_95524.pdf
    File Function: revised version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kirkpatrick, A. Justin & Bennear, Lori S., 2014. "Promoting clean energy investment: An empirical analysis of property assessed clean energy," Journal of Environmental Economics and Management, Elsevier, vol. 68(2), pages 357-375.
    2. Isabela Ferreira Duarte & João Manoel Pinho de Mello & Vinicius Nascimento Carrasco, 2014. "A Década Perdida: 2003 – 2012," Textos para discussão 626, Department of Economics PUC-Rio (Brazil).
    3. Mich�le Belot & Vincent Vandenberghe, 2014. "Evaluating the 'threat' effects of grade repetition: exploiting the 2001 reform by the French-Speaking Community of Belgium," Education Economics, Taylor & Francis Journals, vol. 22(1), pages 73-89, February.
    4. Vincenzo Bove & Leandro Eliay & Ron P Smith, 2014. "The relationship between panel and synthetic control estimators of the effect of civil war," BCAM Working Papers 1406, Birkbeck Centre for Applied Macroeconomics.
    5. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    6. Sarah Bohn & Magnus Lofstrom & Steven Raphael, 2014. "Did the 2007 Legal Arizona Workers Act Reduce the State's Unauthorized Immigrant Population?," The Review of Economics and Statistics, MIT Press, vol. 96(2), pages 258-269, May.
    7. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    8. Jinjarak, Yothin & Noy, Ilan & Zheng, Huanhuan, 2013. "Capital controls in Brazil – Stemming a tide with a signal?," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 2938-2952.
    9. Nikolay Doudchenko & Guido W. Imbens, 2016. "Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis," NBER Working Papers 22791, National Bureau of Economic Research, Inc.
    10. Andreas Billmeier & Tommaso Nannicini, 2013. "Assessing Economic Liberalization Episodes: A Synthetic Control Approach," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 983-1001, July.
    11. Stephen G. Donald & Kevin Lang, 2007. "Inference with Difference-in-Differences and Other Panel Data," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 221-233, May.
    12. Marcos Sanso‐Navarro, 2011. "The Effects on American Foreign Direct Investment in the United Kingdom from Not Adopting the Euro," Journal of Common Market Studies, Wiley Blackwell, vol. 49(2), pages 463-483, March.
    13. Laurent Gobillon & Thierry Magnac, 2016. "Regional Policy Evaluation: Interactive Fixed Effects and Synthetic Controls," The Review of Economics and Statistics, MIT Press, vol. 98(3), pages 535-551, July.
    14. Andreas Billmeier & Tommaso Nannicini, 2009. "Trade Openness and Growth: Pursuing Empirical Glasnost," IMF Staff Papers, Palgrave Macmillan, vol. 56(3), pages 447-475, August.
    15. Jinjarak, Yothin & Noy, Ilan & Zheng, Huanhuan, 2013. "Capital controls in Brazil – Stemming a tide with a signal?," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 2938-2952.
    16. Acemoglu, Daron & Johnson, Simon & Kermani, Amir & Kwak, James & Mitton, Todd, 2016. "The value of connections in turbulent times: Evidence from the United States," Journal of Financial Economics, Elsevier, vol. 121(2), pages 368-391.
    17. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    18. Paolo Pinotti, 2012. "The Economic Costs of Organized Crime: Evidence from Southern Italy," Working Papers 054, "Carlo F. Dondena" Centre for Research on Social Dynamics (DONDENA), Università Commerciale Luigi Bocconi.
    19. Shimeng Liu, 2015. "Spillovers from Universities: Evidence from the Land-Grant Program," Working Paper 9410, USC Lusk Center for Real Estate.
    20. Eduardo Cavallo & Sebastian Galiani & Ilan Noy & Juan Pantano, 2013. "Catastrophic Natural Disasters and Economic Growth," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1549-1561, December.
    21. Peter Hinrichs, 2012. "The Effects of Affirmative Action Bans on College Enrollment, Educational Attainment, and the Demographic Composition of Universities," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 712-722, August.
    22. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    23. Kaul, Ashok & Klößner, Stefan & Pfeifer, Gregor & Schieler, Manuel, 2015. "Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates," MPRA Paper 83790, University Library of Munich, Germany.
    24. Severnini, Edson R., 2014. "The Power of Hydroelectric Dams: Agglomeration Spillovers," IZA Discussion Papers 8082, Institute of Labor Economics (IZA).
    25. Erin O Sills & Diego Herrera & A Justin Kirkpatrick & Amintas Brandão Jr. & Rebecca Dickson & Simon Hall & Subhrendu Pattanayak & David Shoch & Mariana Vedoveto & Luisa Young & Alexander Pfaff, 2015. "Estimating the Impacts of Local Policy Innovation: The Synthetic Control Method Applied to Tropical Deforestation," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-15, July.
    26. William duPont IV & Ilan Noy, 2015. "What Happened to Kobe? A Reassessment of the Impact of the 1995 Earthquake in Japan," Economic Development and Cultural Change, University of Chicago Press, vol. 63(4), pages 777-812.
    27. Calderón Gabriela, 2014. "The Effects of Child Care Provision in Mexico," Working Papers 2014-07, Banco de México.
    28. José G. Montalvo, 2011. "Voting after the Bombings: A Natural Experiment on the Effect of Terrorist Attacks on Democratic Elections," The Review of Economics and Statistics, MIT Press, vol. 93(4), pages 1146-1154, November.
    29. Timothy G. Conley & Christopher R. Taber, 2011. "Inference with "Difference in Differences" with a Small Number of Policy Changes," The Review of Economics and Statistics, MIT Press, vol. 93(1), pages 113-125, February.
    30. Hugo Jales & Thomas H. Kang & Guilherme Stein & Felipe Garcia Ribeiro, 2018. "Measuring the role of the 1959 revolution on Cuba's economic performance," The World Economy, Wiley Blackwell, vol. 41(8), pages 2243-2274, August.
    31. Ando, Michihito, 2015. "Dreams of urbanization: Quantitative case studies on the local impacts of nuclear power facilities using the synthetic control method," Journal of Urban Economics, Elsevier, vol. 85(C), pages 68-85.
    32. Noémi Kreif & Richard Grieve & Dominik Hangartner & Alex James Turner & Silviya Nikolova & Matt Sutton, 2016. "Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units," Health Economics, John Wiley & Sons, Ltd., vol. 25(12), pages 1514-1528, December.
    33. Victor Chernozhukov & Han Hong & Elie Tamer, 2007. "Estimation and Confidence Regions for Parameter Sets in Econometric Models," Econometrica, Econometric Society, vol. 75(5), pages 1243-1284, September.
    34. William duPont IV & Ilan Noy, 2015. "What Happened to Kobe? A Reassessment of the Impact of the 1995 Earthquake in Japan," Economic Development and Cultural Change, University of Chicago Press, vol. 63(4), pages 777-812.
    35. Barone, Guglielmo & Mocetti, Sauro, 2014. "Natural disasters, growth and institutions: A tale of two earthquakes," Journal of Urban Economics, Elsevier, vol. 84(C), pages 52-66.
    36. Amr Hosny, 2012. "Algeria’s Trade with GAFTA Countries: A Synthetic Control Approach," Transition Studies Review, Springer;Central Eastern European University Network (CEEUN), vol. 19(1), pages 35-42, September.
    37. Smith, Brock, 2015. "The resource curse exorcised: Evidence from a panel of countries," Journal of Development Economics, Elsevier, vol. 116(C), pages 57-73.
    38. Dhungana, Sandesh, 2011. "Identifying and evaluating large scale policy interventions : what questions can we answer ?," Policy Research Working Paper Series 5918, The World Bank.
    39. Chan, Ho Fai & Frey, Bruno S. & Gallus, Jana & Torgler, Benno, 2014. "Academic honors and performance," Labour Economics, Elsevier, vol. 31(C), pages 188-204.
    40. Bauhoff, Sebastian, 2014. "The effect of school district nutrition policies on dietary intake and overweight: A synthetic control approach," Economics & Human Biology, Elsevier, vol. 12(C), pages 45-55.
    41. Liu, Shimeng, 2015. "Spillovers from universities: Evidence from the land-grant program," Journal of Urban Economics, Elsevier, vol. 87(C), pages 25-41.
    42. David Powell, 2016. "Synthetic Control Estimation Beyond Case Studies Does the Minimum Wage Reduce Employment?," Working Papers 1142, RAND Corporation.
    43. Mideksa, Torben K., 2013. "The economic impact of natural resources," Journal of Environmental Economics and Management, Elsevier, vol. 65(2), pages 277-289.
    44. David Powell, 2016. "Synthetic Control Estimation Beyond Case Studies Does the Minimum Wage Reduce Employment?," Working Papers WR-1142, RAND Corporation.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    2. Kaul, Ashok & Klößner, Stefan & Pfeifer, Gregor & Schieler, Manuel, 2015. "Synthetic Control Methods: Never Use All Pre-Intervention Outcomes Together With Covariates," MPRA Paper 83790, University Library of Munich, Germany.
    3. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    4. Giulio Grossi & Marco Mariani & Alessandra Mattei & Patrizia Lattarulo & Ozge Oner, 2020. "Direct and spillover effects of a new tramway line on the commercial vitality of peripheral streets. A synthetic-control approach," Papers 2004.05027, arXiv.org, revised Nov 2023.
    5. Kleis, Mischa & Moessinger, Marc-Daniel, 2016. "The long-run effect of fiscal consolidation on economic growth: Evidence from quantitative case studies," ZEW Discussion Papers 16-047, ZEW - Leibniz Centre for European Economic Research, revised 2016.
    6. Samuel Verevis & Murat Üngör, 2021. "What has New Zealand gained from The FTA with China?: Two counterfactual analyses†," Scottish Journal of Political Economy, Scottish Economic Society, vol. 68(1), pages 20-50, February.
    7. Emery, Thomas & Mélon, Lela & Spruk, Rok, 2023. "Does e-procurement matter for economic growth? Subnational evidence from Australia," The Quarterly Review of Economics and Finance, Elsevier, vol. 89(C), pages 318-334.
    8. Klößner, Stefan & Pfeifer, Gregor, 2015. "Synthesizing Cash for Clunkers: Stabilizing the Car Market, Hurting the Environment," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113207, Verein für Socialpolitik / German Economic Association.
    9. Irene Botosaru & Bruno Ferman, 2019. "On the role of covariates in the synthetic control method," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 117-130.
    10. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2021. "Decoupling synthetic control methods to ensure stability, accuracy and meaningfulness," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 12(4), pages 549-584, December.
    11. Maïmouna DIAKITE & Jean-François BRUN & Souleymane DIARRA & Nasser ARY TANIMOUNE, 2017. "The effects of tax coordination on the tax revenue mobilization in West African Economic and Monetary Union (WAEMU)," Working Papers 201712, CERDI.
    12. Pier Basaglia & Sophie Behr & Moritz A. Drupp, 2023. "De-Fueling Externalities: How Tax Salience and Fuel Substitution Mediate Climate and Health Benefits," Discussion Papers of DIW Berlin 2041, DIW Berlin, German Institute for Economic Research.
    13. Pier Basaglia & Sophie M. Behr & Moritz A. Drupp, 2023. "De-Fueling Externalities: Causal Effects of Fuel Taxation and Mediating Mechanisms for Reducing Climate and Pollution Costs," CESifo Working Paper Series 10508, CESifo.
    14. Matej Opatrny, 2021. "The impact of the Brexit vote on UK financial markets: a synthetic control method approach," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 48(2), pages 559-587, May.
    15. David Powell, 2016. "Synthetic Control Estimation Beyond Case Studies Does the Minimum Wage Reduce Employment?," Working Papers WR-1142, RAND Corporation.
    16. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2020. "Ensuring Stability, Accuracy and Meaningfulness in Synthetic Control Methods: The Regularized SHAP-Distance Method," IREA Working Papers 202005, University of Barcelona, Research Institute of Applied Economics, revised Apr 2020.
    17. Carvalho, Carlos & Masini, Ricardo & Medeiros, Marcelo C., 2018. "ArCo: An artificial counterfactual approach for high-dimensional panel time-series data," Journal of Econometrics, Elsevier, vol. 207(2), pages 352-380.
    18. Eliason, Paul & Lutz, Byron, 2018. "Can fiscal rules constrain the size of government? An analysis of the “crown jewel” of tax and expenditure limitations," Journal of Public Economics, Elsevier, vol. 166(C), pages 115-144.
    19. Joseph Cummins & Brock Smith & Douglas L. Miller & David Eliot Simon, 2023. "Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator," Working papers 2023-07, University of Connecticut, Department of Economics.
    20. Bibek Adhikari & Romain Duval & Bingjie Hu & Prakash Loungani, 2018. "Can Reform Waves Turn the Tide? Some Case Studies using the Synthetic Control Method," Open Economies Review, Springer, vol. 29(4), pages 879-910, September.

    More about this item

    Keywords

    synthetic control; difference-in-differences; linear factor model; inference; permutation test;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:73982. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.