IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1912.03290.html
   My bibliography  Save this paper

Synthetic Controls with Staggered Adoption

Author

Listed:
  • Eli Ben-Michael
  • Avi Feller
  • Jesse Rothstein

Abstract

Staggered adoption of policies by different units at different times creates promising opportunities for observational causal inference. Estimation remains challenging, however, and common regression methods can give misleading results. A promising alternative is the synthetic control method (SCM), which finds a weighted average of control units that closely balances the treated unit's pre-treatment outcomes. In this paper, we generalize SCM, originally designed to study a single treated unit, to the staggered adoption setting. We first bound the error for the average effect and show that it depends on both the imbalance for each treated unit separately and the imbalance for the average of the treated units. We then propose "partially pooled" SCM weights to minimize a weighted combination of these measures; approaches that focus only on balancing one of the two components can lead to bias. We extend this approach to incorporate unit-level intercept shifts and auxiliary covariates. We assess the performance of the proposed method via extensive simulations and apply our results to the question of whether teacher collective bargaining leads to higher school spending, finding minimal impacts. We implement the proposed method in the augsynth R package.

Suggested Citation

  • Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2019. "Synthetic Controls with Staggered Adoption," Papers 1912.03290, arXiv.org, revised Jan 2021.
  • Handle: RePEc:arx:papers:1912.03290
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1912.03290
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Taisuke Otsu & Yoshiyasu Rai, 2017. "Bootstrap Inference of Matching Estimators for Average Treatment Effects," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1720-1732, October.
    2. Azeem M. Shaikh & Panos Toulis, 2021. "Randomization Tests in Observational Studies With Staggered Adoption of Treatment," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1835-1848, October.
    3. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    4. Kosuke Imai & David A. van Dyk, 2004. "Causal Inference With General Treatment Regimes: Generalizing the Propensity Score," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 854-866, January.
    5. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    6. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(1), pages 1-19.
    7. Imai, Kosuke & Kim, In Song, 2021. "On the Use of Two-Way Fixed Effects Regression Models for Causal Inference with Panel Data," Political Analysis, Cambridge University Press, vol. 29(3), pages 405-415, July.
    8. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2021. "Matrix Completion Methods for Causal Panel Data Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1716-1730, October.
    9. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    10. Brantly Callaway & Pedro H. C. Sant'Anna, 2018. "Difference-in-Differences with Multiple Time Periods and an Application on the Minimum Wage and Employment," DETU Working Papers 1804, Department of Economics, Temple University.
    11. C. Kirabo Jackson & Jonah E. Rockoff & Douglas O. Staiger, 2014. "Teacher Effects and Teacher-Related Policies," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 801-825, August.
    12. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    13. Alberto Abadie & Guido W. Imbens, 2011. "Bias-Corrected Matching Estimators for Average Treatment Effects," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 29(1), pages 1-11, January.
    14. Gary King & Christopher Lucas & Richard A. Nielsen, 2017. "The Balance‐Sample Size Frontier in Matching Methods for Causal Inference," American Journal of Political Science, John Wiley & Sons, vol. 61(2), pages 473-489, April.
    15. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    16. Caroline Minter Hoxby, 1996. "How Teachers' Unions Affect Education Production," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 111(3), pages 671-718.
    17. Michael W. Robbins & Jessica Saunders & Beau Kilmer, 2017. "A Framework for Synthetic Control Methods With High-Dimensional, Micro-Level Data: Evaluating a Neighborhood-Specific Crime Intervention," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(517), pages 109-126, January.
    18. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    19. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    20. Andrew Goodman-Bacon, 2018. "Difference-in-Differences with Variation in Treatment Timing," NBER Working Papers 25018, National Bureau of Economic Research, Inc.
    21. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    22. Ariella Kahn-Lang & Kevin Lang, 2020. "The Promise and Pitfalls of Differences-in-Differences: Reflections on 16 and Pregnant and Other Applications," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(3), pages 613-620, July.
    23. Dube, Arindrajit & Zipperer, Ben, 2015. "Pooling Multiple Case Studies Using Synthetic Controls: An Application to Minimum Wage Policies," IZA Discussion Papers 8944, Institute of Labor Economics (IZA).
    24. Agustina S. Paglayan, 2019. "Public‐Sector Unions and the Size of Government," American Journal of Political Science, John Wiley & Sons, vol. 63(1), pages 21-36, January.
    25. Noémi Kreif & Richard Grieve & Dominik Hangartner & Alex James Turner & Silviya Nikolova & Matt Sutton, 2016. "Examination of the Synthetic Control Method for Evaluating Health Policies with Multiple Treated Units," Health Economics, John Wiley & Sons, Ltd., vol. 25(12), pages 1514-1528, December.
    26. Jonathan Roth, 2018. "Should We Adjust for the Test for Pre-trends in Difference-in-Difference Designs?," Papers 1804.01208, arXiv.org, revised May 2018.
    27. Xu, Yiqing, 2017. "Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models," Political Analysis, Cambridge University Press, vol. 25(1), pages 57-76, January.
    28. James E. Pustejovsky & Elizabeth Tipton, 2018. "Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 672-683, October.
    29. Jaap H. Abbring & Gerard J. van den Berg, 2003. "The Nonparametric Identification of Treatment Effects in Duration Models," Econometrica, Econometric Society, vol. 71(5), pages 1491-1517, September.
    30. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    31. Michael F. Lovenheim, 2009. "The Effect of Teachers' Unions on Education Production: Evidence from Union Election Certifications in Three Midwestern States," Journal of Labor Economics, University of Chicago Press, vol. 27(4), pages 525-587, October.
    32. Jianfei Cao & Shirley Lu, 2019. "Synthetic Control Inference for Staggered Adoption: Estimating the Dynamic Effects of Board Gender Diversity Policies," Papers 1912.06320, arXiv.org.
    33. Alberto Abadie & Jérémy L’Hour, 2021. "A Penalized Synthetic Control Estimator for Disaggregated Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1817-1834, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Peri, Giovanni & Rury, Derek & Wiltshire, Justin C., 2020. "The Economic Impact of Migrants from Hurricane Maria," IZA Discussion Papers 13049, Institute of Labor Economics (IZA).
    2. Ron Berman & Ayelet Israeli, 2022. "The Value of Descriptive Analytics: Evidence from Online Retailers," Marketing Science, INFORMS, vol. 41(6), pages 1074-1096, November.
    3. Manuel Funke & Moritz Schularick & Christoph Trebesch, 2023. "Populist Leaders and the Economy," American Economic Review, American Economic Association, vol. 113(12), pages 3249-3288, December.
    4. Tomasz Serwach, 2022. "The European Union and within-country income inequalities. The case of the New Member States," Working Papers hal-03548416, HAL.
    5. Porreca, Zachary, 2022. "Synthetic difference-in-differences estimation with staggered treatment timing," Economics Letters, Elsevier, vol. 220(C).
    6. Isabelle Chort & Berk Öktem, 2023. "Agricultural shocks, coping policies and deforestation: evidence from the coffee leaf rust epidemic in mexico," Working Papers hal-03715600, HAL.
    7. Bennett, Magdalena, 2021. "All things equal? Heterogeneity in policy effectiveness against COVID-19 spread in chile," World Development, Elsevier, vol. 137(C).
    8. Wagner, Gary A. & Rork, Jonathan C., 2023. "Does state tax reciprocity affect interstate commuting? Evidence from a natural experiment," Regional Science and Urban Economics, Elsevier, vol. 102(C).
    9. Nadine McCloud, 2022. "Does domestic investment respond to inflation targeting? A synthetic control investigation," International Economics, CEPII research center, issue 169, pages 98-134.
    10. Kim, Youngho, 2023. "Payments for Ecosystem Services Programs and Climate Change Adaptation in Agriculture," 2023 Annual Meeting, July 23-25, Washington D.C. 335971, Agricultural and Applied Economics Association.
    11. Armstrong, Christopher & Kepler, John D. & Samuels, Delphine & Taylor, Daniel, 2022. "Causality redux: The evolution of empirical methods in accounting research and the growth of quasi-experiments," Journal of Accounting and Economics, Elsevier, vol. 74(2).
    12. Roberta Di Stefano & Giovanni Mellace, 2020. "The inclusive synthetic control method," Working Papers 21/20, Sapienza University of Rome, DISS.
    13. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    14. Guihua Wang, 2022. "Stay at home to stay safe: Effectiveness of stay‐at‐home orders in containing the COVID‐19 pandemic," Production and Operations Management, Production and Operations Management Society, vol. 31(5), pages 2289-2305, May.
    15. Rong J. B. Zhu, 2023. "Synthetic Regressing Control Method," Papers 2306.02584, arXiv.org, revised Oct 2023.
    16. Hideki Shimada & Kenji Asano & Yu Nagai & Akito Ozawa, 2022. "Assessing the Impact of Offshore Wind Power Deployment on Fishery: A Synthetic Control Approach," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 83(3), pages 791-829, November.
    17. Xingyu Li & Yan Shen & Qiankun Zhou, 2022. "Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects," Papers 2202.12078, arXiv.org.
    18. Giulio Grossi, 2023. "The policy is always greener: impact heterogeneity of Covid-19 vaccination lotteries in the US," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 32(4), pages 1351-1375, October.
    19. Ursula Muench & Armin Nassehi & Joe Kaeser & Knut Bergmann & Matthias Diermeier & Florian Dorn & David Gstrein & Florian Neumeier & Manuel Funke & Moritz Schularick & Christoph Trebesch & Kerim Peren , 2024. "Wohlstand in Gefahr? Ursachen und Folgen von Populismus," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 77(03), pages 03-32, March.
    20. Matias D. Cattaneo & Yingjie Feng & Filippo Palomba & Rocio Titiunik, 2022. "scpi: Uncertainty Quantification for Synthetic Control Methods," Papers 2202.05984, arXiv.org, revised Oct 2022.
    21. Beltran-Silva, Francisco, 2023. "Fighting against hunger: A country-wide intervention and its impact on birth outcomes," World Development, Elsevier, vol. 165(C).
    22. Hodor, Michal, 2021. "Family health spillovers: evidence from the RAND health insurance experiment," Journal of Health Economics, Elsevier, vol. 79(C).
    23. Jiafeng Chen, 2023. "Synthetic Control as Online Linear Regression," Econometrica, Econometric Society, vol. 91(2), pages 465-491, March.
    24. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    2. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    3. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    4. Jianfei Cao & Shirley Lu, 2019. "Synthetic Control Inference for Staggered Adoption: Estimating the Dynamic Effects of Board Gender Diversity Policies," Papers 1912.06320, arXiv.org.
    5. Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org, revised Aug 2022.
    6. Tomasz Serwach, 2022. "The European Union and within-country income inequalities. The case of the New Member States," Working Papers hal-03548416, HAL.
    7. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    8. Roy Cerqueti & Raffaella Coppier & Alessandro Girardi & Marco Ventura, 2022. "The sooner the better: lives saved by the lockdown during the COVID-19 outbreak. The case of Italy [Using synthetic controls: Feasibility, data requirements, and methodological aspects]," The Econometrics Journal, Royal Economic Society, vol. 25(1), pages 46-70.
    9. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    10. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    11. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    12. Luis Costa & Vivek F. Farias & Patricio Foncea & Jingyuan (Donna) Gan & Ayush Garg & Ivo Rosa Montenegro & Kumarjit Pathak & Tianyi Peng & Dusan Popovic, 2023. "Generalized Synthetic Control for TestOps at ABI: Models, Algorithms, and Infrastructure," Interfaces, INFORMS, vol. 53(5), pages 336-349, September.
    13. Kim, Youngho, 2023. "Payments for Ecosystem Services Programs and Climate Change Adaptation in Agriculture," 2023 Annual Meeting, July 23-25, Washington D.C. 335971, Agricultural and Applied Economics Association.
    14. Guido W. Imbens & Davide Viviano, 2023. "Identification and Inference for Synthetic Controls with Confounding," Papers 2312.00955, arXiv.org.
    15. Joakim Weill, 2023. "Flood Risk Mapping and the Distributional Impacts of Climate Information," Working Papers 2023.10, FAERE - French Association of Environmental and Resource Economists.
    16. Joakim A. Weill, 2023. "Flood Risk Mapping and the Distributional Impacts of Climate Information," Finance and Economics Discussion Series 2023-066, Board of Governors of the Federal Reserve System (U.S.).
    17. Viviano, Davide & Bradic, Jelena, 2023. "Synthetic Learner: Model-free inference on treatments over time," Journal of Econometrics, Elsevier, vol. 234(2), pages 691-713.
    18. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    19. Tomasz Serwach, 2023. "The European Union and within‐country income inequalities. The case of the new member states," The World Economy, Wiley Blackwell, vol. 46(7), pages 1890-1939, July.
    20. Lea Bottmer & Guido Imbens & Jann Spiess & Merrill Warnick, 2021. "A Design-Based Perspective on Synthetic Control Methods," Papers 2101.09398, arXiv.org, revised Jul 2023.

    More about this item

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • I21 - Health, Education, and Welfare - - Education - - - Analysis of Education
    • J5 - Labor and Demographic Economics - - Labor-Management Relations, Trade Unions, and Collective Bargaining

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1912.03290. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.