IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1904.01490.html
   My bibliography  Save this paper

Synthetic learner: model-free inference on treatments over time

Author

Listed:
  • Davide Viviano
  • Jelena Bradic

Abstract

Understanding of the effect of a particular treatment or a policy pertains to many areas of interest -- ranging from political economics, marketing to health-care and personalized treatment studies. In this paper, we develop a non-parametric, model-free test for detecting the effects of treatment over time that extends widely used Synthetic Control tests. The test is built on counterfactual predictions arising from many learning algorithms. In the Neyman-Rubin potential outcome framework with possible carry-over effects, we show that the proposed test is asymptotically consistent for stationary, beta mixing processes. We do not assume that class of learners captures the correct model necessarily. We also discuss estimates of the average treatment effect, and we provide regret bounds on the predictive performance. To the best of our knowledge, this is the first set of results that allow for example any Random Forest to be useful for provably valid statistical inference in the Synthetic Control setting. In experiments, we show that our Synthetic Learner is substantially more powerful than classical methods based on Synthetic Control or Difference-in-Differences, especially in the presence of non-linear outcome models.

Suggested Citation

  • Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org.
  • Handle: RePEc:arx:papers:1904.01490
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1904.01490
    File Function: Latest version
    Download Restriction: no

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Kaspar Wüthrich & Yu Zhu, 2017. "An exact and robust conformal inference method for counterfactual and synthetic controls," CeMMAP working papers CWP62/17, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2018. "The Augmented Synthetic Control Method," Papers 1811.04170, arXiv.org, revised Nov 2019.
    3. Cheng Hsiao & H. Steve Ching & Shui Ki Wan, 2012. "A Panel Data Approach For Program Evaluation: Measuring The Benefits Of Political And Economic Integration Of Hong Kong With Mainland China," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(5), pages 705-740, August.
    4. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    5. repec:taf:jnlasa:v:113:y:2018:i:523:p:1228-1242 is not listed on IDEAS
    6. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    7. A. Belloni & V. Chernozhukov & I. Fernández‐Val & C. Hansen, 2017. "Program Evaluation and Causal Inference With High‐Dimensional Data," Econometrica, Econometric Society, vol. 85, pages 233-298, January.
    8. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2017. "Matrix Completion Methods for Causal Panel Data Models," Papers 1710.10251, arXiv.org, revised Sep 2018.
    9. Abbring, Jaap H. & Heckman, James J., 2007. "Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 72, Elsevier.
    10. Susan Athey & Guido W. Imbens & Stefan Wager, 2018. "Approximate residual balancing: debiased inference of average treatment effects in high dimensions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(4), pages 597-623, September.
    11. Xu, Yiqing, 2017. "Generalized Synthetic Control Method: Causal Inference with Interactive Fixed Effects Models," Political Analysis, Cambridge University Press, vol. 25(1), pages 57-76, January.
    12. Ashesh Rambachan & Neil Shephard, 2019. "A Nonparametric Dynamic Causal Model for Macroeconometrics," Papers 1903.01637, arXiv.org.
    13. repec:taf:jnlbes:v:34:y:2016:i:4:p:661-672 is not listed on IDEAS
    14. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
    15. Jinyong Hahn & Ruoyao Shi, 2017. "Synthetic Control and Inference," Econometrics, MDPI, Open Access Journal, vol. 5(4), pages 1-12, November.
    16. Craig Garthwaite & Tal Gross & Matthew J. Notowidigdo, 2014. "Public Health Insurance, Labor Supply, and Employment Lock," The Quarterly Journal of Economics, Oxford University Press, vol. 129(2), pages 653-696.
    17. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    18. repec:taf:jnlasa:v:113:y:2018:i:523:p:1112-1121 is not listed on IDEAS
    19. Nicolo Cesa Bianchi & Gábor Lugosi, 1998. "On prediction of individual sequences," Economics Working Papers 324, Department of Economics and Business, Universitat Pompeu Fabra.
    20. Theis Lange & Anders Rahbek & Søren Tolver Jensen, 2011. "Estimation and Asymptotic Inference in the AR-ARCH Model," Econometric Reviews, Taylor & Francis Journals, vol. 30(2), pages 129-153.
    21. Max H. Farrell & Tengyuan Liang & Sanjog Misra, 2018. "Deep Neural Networks for Estimation and Inference," Papers 1809.09953, arXiv.org, revised Sep 2019.
    22. Victor Chernozhukov & Kaspar Wuthrich & Yinchu Zhu, 2018. "Practical and robust $t$-test based inference for synthetic control and related methods," Papers 1812.10820, arXiv.org, revised Jun 2019.
    23. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    24. Knaus, Michael C. & Lechner, Michael & Strittmatter, Anthony, 2018. "Machine Learning Estimation of Heterogeneous Causal Effects: Empirical Monte Carlo Evidence," IZA Discussion Papers 12039, Institute of Labor Economics (IZA).
    25. White, Halbert & Domowitz, Ian, 1984. "Nonlinear Regression with Dependent Observations," Econometrica, Econometric Society, vol. 52(1), pages 143-161, January.
    26. Kolstad, Jonathan T. & Kowalski, Amanda E., 2012. "The impact of health care reform on hospital and preventive care: Evidence from Massachusetts," Journal of Public Economics, Elsevier, vol. 96(11), pages 909-929.
    27. repec:bpj:causin:v:6:y:2018:i:2:p:26:n:1 is not listed on IDEAS
    28. Sarah Abraham & Liyang Sun, 2018. "Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effects," Papers 1804.05785, arXiv.org.
    29. Michael Anderson & Carlos Dobkin & Tal Gross, 2012. "The Effect of Health Insurance Coverage on the Use of Medical Services," American Economic Journal: Economic Policy, American Economic Association, vol. 4(1), pages 1-27, February.
    30. repec:wly:amposc:v:59:y:2015:i:2:p:495-510 is not listed on IDEAS
    31. Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(1), pages 17-39, February.
    32. repec:cup:apsrev:v:112:y:2018:i:04:p:1067-1082_00 is not listed on IDEAS
    33. Sharon K. Long & Karen Stockley & Alshadye Yemane, 2009. "Another Look at the Impacts of Health Reform in Massachusetts: Evidence Using New Data and a Stronger Model," American Economic Review, American Economic Association, vol. 99(2), pages 508-511, May.
    34. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    Full references (including those not matched with items on IDEAS)

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1904.01490. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (arXiv administrators). General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.