IDEAS home Printed from
   My bibliography  Save this paper

Placebo Tests for Synthetic Controls


  • Ferman, Bruno
  • Pinto, Cristine


The synthetic control (SC) method has been recently proposed as an alternative to estimate treatment effects in comparative case studies. An important feature of the SC method is the inferential procedures based on placebo studies, suggested in Abadie et al. (2010). In this paper, we evaluate the statistical properties of these inferential techniques. We first show that the graphical analysis with placebos can be misleading, as placebo runs with lower expected squared prediction errors would still be considered in the analysis. Then we show that a test based on the the post/pre-intervention mean squared prediction error, as suggested in Abadie et al. (2010), ameliorates this problem. However, we show that such test can still have some size distortions, even if we consider a case in which the test statistic has the same marginal distribution for all placebo runs. Finally, we show that the fact that the SC weights are estimated can lead to important additional size distortions.

Suggested Citation

  • Ferman, Bruno & Pinto, Cristine, 2017. "Placebo Tests for Synthetic Controls," MPRA Paper 78079, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:78079

    Download full text from publisher

    File URL:
    File Function: original version
    Download Restriction: no

    References listed on IDEAS

    1. Bruno Ferman & Cristine Pinto, 2019. "Inference in Differences-in-Differences with Few Treated Groups and Heteroskedasticity," The Review of Economics and Statistics, MIT Press, vol. 101(3), pages 452-467, July.
    2. repec:fgv:eesptd:411 is not listed on IDEAS
    3. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    4. Nikolay Doudchenko & Guido W. Imbens, 2016. "Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis," NBER Working Papers 22791, National Bureau of Economic Research, Inc.
    5. repec:aea:jecper:v:31:y:2017:i:2:p:3-32 is not listed on IDEAS
    6. Carlos Viana de Carvalho & Ricardo Masini & Marcelo Cunha Medeiros, 2016. "The perils of Counterfactual Analysis with Integrated Processes," Textos para discussão 654, Department of Economics PUC-Rio (Brazil).
    7. Ferman, Bruno & Pinto, Cristine Campos de Xavier, 2016. "Revisiting the synthetic control estimator," Textos para discussão 421, FGV EESP - Escola de Economia de São Paulo, Fundação Getulio Vargas (Brazil).
    8. David Powell, 2016. "Synthetic Control Estimation Beyond Case Studies Does the Minimum Wage Reduce Employment?," Working Papers 1142, RAND Corporation.
    9. Jinyong Hahn & Ruoyao Shi, 2017. "Synthetic Control and Inference," Econometrics, MDPI, Open Access Journal, vol. 5(4), pages 1-12, November.
    10. David Powell, 2016. "Synthetic Control Estimation Beyond Case Studies Does the Minimum Wage Reduce Employment?," Working Papers WR-1142, RAND Corporation.
    11. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Jianfei Cao & Connor Dowd, 2019. "Estimation and Inference for Synthetic Control Methods with Spillover Effects," Papers 1902.07343,, revised Nov 2019.
    2. Felipe Filgueiras, Elias Cavalcante-Filho, Rodrigo de Losso, José Roberto Savoia, 2019. "Law Change in a Regulated Sector Impacts Other Regulated Sectors: Evidence from Brazil," Working Papers, Department of Economics 2019_27, University of São Paulo (FEA-USP).
    3. Ferman, Bruno, 2017. "Matching Estimators with Few Treated and Many Control Observations," MPRA Paper 78940, University Library of Munich, Germany.
    4. Nadler, Carl & Allegretto, Sylvia & Godoey, Anna & Reich, Michael, 2019. "Are Local Minimum Wages Too High? Working Paper #102-19," Institute for Research on Labor and Employment, Working Paper Series qt7xt8716f, Institute of Industrial Relations, UC Berkeley.
    5. Klößner, Stefan & Pfeifer, Gregor, 2015. "Synthesizing Cash for Clunkers: Stabilizing the Car Market, Hurting the Environment," Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113207, Verein für Socialpolitik / German Economic Association.
    6. Victor Chernozhukov & Kaspar Wüthrich & Yu Zhu, 2017. "An exact and robust conformal inference method for counterfactual and synthetic controls," CeMMAP working papers CWP62/17, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    7. Victor Chernozhukov & Kaspar Wuthrich & Yinchu Zhu, 2018. "Practical and robust $t$-test based inference for synthetic control and related methods," Papers 1812.10820,, revised Jun 2019.
    8. Julia Bluszcz & Marica Valente, 2019. "The War in Europe: Economic Costs of the Ukrainian Conflict," Discussion Papers of DIW Berlin 1804, DIW Berlin, German Institute for Economic Research.

    More about this item


    synthetic control; difference-in-differences; linear factor model; inference; permutation test;

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:78079. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Joachim Winter). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.