IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2603.19211.html

Synthetic Control Misconceptions: Recommendations for Practice

Author

Listed:
  • Robert Pickett
  • Jennifer Hill
  • Sarah Cowan

Abstract

To estimate the causal effect of an intervention, researchers need to identify a control group that represents what might have happened to the treatment group in the absence of that intervention. This is challenging without a randomized experiment and further complicated when few units (possibly only one) are treated. Nevertheless, when data are available on units over time, synthetic control (SC) methods provide an opportunity to construct a valid comparison by differentially weighting control units that did not receive the treatment so that their resulting pre-treatment trajectory is similar to that of the treated unit. The hope is that this weighted ``pseudo-counterfactual" can serve as a valid counterfactual in the post-treatment time period. Since its origin twenty years ago, SC has been used over 5,000 times in the literature (Web of Science, December 2025), leading to a proliferation of descriptions of the method and guidance on proper usage that is not always accurate and does not always align with what the original developers appear to have intended. As such, a number of accepted pieces of wisdom have arisen: (1) SC is robust to various implementations; (2) covariates are unnecessary, and (3) pre-treatment prediction error should guide model selection. We describe each in detail and conduct simulations that suggest, both for standard and alternative implementations of SC, that these purported truths are not supported by empirical evidence and thus actually represent misconceptions about best practice. Instead of relying on these misconceptions, we offer practical advice for more cautious implementation and interpretation of results.

Suggested Citation

  • Robert Pickett & Jennifer Hill & Sarah Cowan, 2026. "Synthetic Control Misconceptions: Recommendations for Practice," Papers 2603.19211, arXiv.org.
  • Handle: RePEc:arx:papers:2603.19211
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2603.19211
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Propheter, Geoffrey, 2020. "The effect of a new sports facility on property development: Evidence from building permits and a localized synthetic control," Journal of Regional Analysis and Policy, Mid-Continent Regional Science Association, vol. 50(01), December.
    2. Richard Dorsett, 2021. "A Bayesian structural time series analysis of the effect of basic income on crime: Evidence from the Alaska Permanent Fund," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 179-200, January.
    3. Xu, Yiqing, 2017. "Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models," Political Analysis, Cambridge University Press, vol. 25(1), pages 57-76, January.
    4. Ashok Kaul & Stefan Klößner & Gregor Pfeifer & Manuel Schieler, 2022. "Standard Synthetic Control Methods: The Case of Using All Preintervention Outcomes Together With Covariates," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 40(3), pages 1362-1376, June.
    5. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    6. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    7. Matej Opatrny, 2021. "The impact of the Brexit vote on UK financial markets: a synthetic control method approach," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 48(2), pages 559-587, May.
    8. John J. Donohue & Abhay Aneja & Kyle D. Weber, 2019. "Right‐to‐Carry Laws and Violent Crime: A Comprehensive Assessment Using Panel Data and a State‐Level Synthetic Control Analysis," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 16(2), pages 198-247, June.
    9. Damon Jones & Ioana Marinescu, 2022. "The Labor Market Impacts of Universal and Permanent Cash Transfers: Evidence from the Alaska Permanent Fund," American Economic Journal: Economic Policy, American Economic Association, vol. 14(2), pages 315-340, May.
    10. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    11. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    12. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    13. Pekka Malo & Juha Eskelinen & Xun Zhou & Timo Kuosmanen, 2024. "Computing Synthetic Controls Using Bilevel Optimization," Computational Economics, Springer;Society for Computational Economics, vol. 64(2), pages 1113-1136, August.
    14. Irene Botosaru & Bruno Ferman, 2019. "On the role of covariates in the synthetic control method," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 117-130.
    15. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Garoupa, Nuno & Spruk, Rok, 2025. "Populist constitutional backsliding and judicial independence: Evidence from Türkiye," International Review of Law and Economics, Elsevier, vol. 84(C).
    2. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    3. Rok Spruk, 2026. "Confrontation with the West and Long-Run Economic and Institutional Outcomes: Evidence from Iran," Papers 2602.03231, arXiv.org.
    4. Fry, Joseph, 2024. "A method of moments approach to asymptotically unbiased Synthetic Controls," Journal of Econometrics, Elsevier, vol. 244(1).
    5. Joseph Fry, 2023. "A Method of Moments Approach to Asymptotically Unbiased Synthetic Controls," Papers 2312.01209, arXiv.org, revised Mar 2024.
    6. Gabriel, Ricardo Duque & Pessoa, Ana Sofia, 2024. "Adopting the euro: A synthetic control approach," European Journal of Political Economy, Elsevier, vol. 83(C).
    7. Cummins Joseph & Miller Douglas L. & Smith Brock & Simon David, 2024. "Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator," Journal of Econometric Methods, De Gruyter, vol. 13(1), pages 67-95, January.
    8. Tomasz Serwach, 2023. "The European Union and within‐country income inequalities. The case of the new member states," The World Economy, Wiley Blackwell, vol. 46(7), pages 1890-1939, July.
    9. Kuosmanen, Timo & Zhou, Xun & Eskelinen, Juha & Malo, Pekka, 2021. "Design Flaw of the Synthetic Control Method," MPRA Paper 106328, University Library of Munich, Germany.
    10. Tomasz Serwach, 2022. "The European Union and within-country income inequalities. The case of the New Member States," Working Papers hal-03548416, HAL.
    11. Pier Basaglia & Sophie M. Behr & Moritz A. Drupp, 2023. "De-Fueling Externalities: How Tax Salience and Fuel Substitution Mediate Climate and Health Benefits," Discussion Papers of DIW Berlin 2041, DIW Berlin, German Institute for Economic Research.
    12. Daniel Albalate & Germà Bel & Ferran A. Mazaira-Font, 2021. "Decoupling synthetic control methods to ensure stability, accuracy and meaningfulness," SERIEs: Journal of the Spanish Economic Association, Springer;Spanish Economic Association, vol. 12(4), pages 549-584, December.
    13. Klößner, Stefan & Pfeifer, Gregor, 2015. "Synthesizing Cash for Clunkers: Stabilizing the Car Market, Hurting the Environment," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113207, Verein für Socialpolitik / German Economic Association.
    14. Piero Basaglia & Sophie M. Behr & Moritz A. Drupp, 2025. "Fuel Taxation and Environmental Externalities: Evidence from the World’s Largest Environmental Tax Reform," CESifo Working Paper Series 11949, CESifo.
    15. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    16. Brett Parker, 2021. "Death Penalty Statutes and Murder Rates: Evidence From Synthetic Controls," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 18(3), pages 488-533, September.
    17. Pier Basaglia & Sophie M. Behr & Moritz A. Drupp & Piero Basaglia, 2023. "De-Fueling Externalities: Causal Effects of Fuel Taxation and Mediating Mechanisms for Reducing Climate and Pollution Costs," CESifo Working Paper Series 10508, CESifo.
    18. Nuno Garoupa & Rok Spruk, 2025. "Revolutions as structural breaks: the long-term economic and institutional consequences of the 1979 Iranian Revolution," Constitutional Political Economy, Springer, vol. 36(3), pages 273-301, September.
    19. Jaroslaw Kantorowicz & Rok Spruk, 2024. "Using synthetic control method to estimate the growth effects of economic liberalisation: Evidence from transition economies," The World Economy, Wiley Blackwell, vol. 47(6), pages 2332-2360, June.
    20. John Charles Bradbury, 2022. "The impact of sports stadiums on localized commercial activity: Evidence from a Business Improvement District," Journal of Regional Science, Wiley Blackwell, vol. 62(1), pages 194-217, January.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2603.19211. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.