IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2207.14481.html
   My bibliography  Save this paper

Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data

Author

Listed:
  • Dennis Shen
  • Peng Ding
  • Jasjeet Sekhon
  • Bin Yu

Abstract

A central goal in social science is to evaluate the causal effect of a policy. One dominant approach is through panel data analysis in which the behaviors of multiple units are observed over time. The information across time and space motivates two general approaches: (i) horizontal regression (i.e., unconfoundedness), which exploits time series patterns, and (ii) vertical regression (e.g., synthetic controls), which exploits cross-sectional patterns. Conventional wisdom states that the two approaches are fundamentally different. We establish this position to be partly false for estimation but generally true for inference. In particular, we prove that both approaches yield identical point estimates under several standard settings. For the same point estimate, however, each approach quantifies uncertainty with respect to a distinct estimand. In turn, the confidence interval developed for one estimand may have incorrect coverage for another. This emphasizes that the source of randomness that researchers assume has direct implications for the accuracy of inference.

Suggested Citation

  • Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
  • Handle: RePEc:arx:papers:2207.14481
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2207.14481
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    2. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    3. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    4. Muhummad Amjad & Vishal Misra & Devavrat Shah & Dennis Shen, 2019. "mRSC: Multi-dimensional Robust Synthetic Control," Papers 1905.06400, arXiv.org, revised Sep 2019.
    5. Michael M. Bechtel & Jens Hainmueller, 2011. "How Lasting Is Voter Gratitude? An Analysis of the Short‐ and Long‐Term Electoral Returns to Beneficial Policy," American Journal of Political Science, John Wiley & Sons, vol. 55(4), pages 852-868, October.
    6. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    7. Susan Athey & Mohsen Bayati & Nikolay Doudchenko & Guido Imbens & Khashayar Khosravi, 2021. "Matrix Completion Methods for Causal Panel Data Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1716-1730, October.
    8. Lea Bottmer & Guido Imbens & Jann Spiess & Merrill Warnick, 2021. "A Design-Based Perspective on Synthetic Control Methods," Papers 2101.09398, arXiv.org, revised Jul 2023.
    9. Kathleen T. Li, 2020. "Statistical Inference for Average Treatment Effects Estimated by Synthetic Control Methods," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 115(532), pages 2068-2083, December.
    10. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    11. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2011. "Synth: An R Package for Synthetic Control Methods in Comparative Case Studies," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 42(i13).
    12. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    13. Anish Agarwal & Devavrat Shah & Dennis Shen & Dogyoon Song, 2021. "On Robustness of Principal Component Regression," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1731-1745, October.
    14. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    15. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2021. "An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1849-1864, October.
    16. Matias D. Cattaneo & Yingjie Feng & Rocio Titiunik, 2021. "Prediction Intervals for Synthetic Control Methods," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1865-1880, October.
    17. Li, Kathleen T. & Bell, David R., 2017. "Estimation of average treatment effects with panel data: Asymptotic theory and implementation," Journal of Econometrics, Elsevier, vol. 197(1), pages 65-75.
    18. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    19. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    20. Bruno Ferman & Cristine Pinto, 2021. "Synthetic controls with imperfect pretreatment fit," Quantitative Economics, Econometric Society, vol. 12(4), pages 1197-1221, November.
    21. Alberto Abadie, 2021. "Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects," Journal of Economic Literature, American Economic Association, vol. 59(2), pages 391-425, June.
    22. Nikolay Doudchenko & Guido W. Imbens, 2016. "Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis," NBER Working Papers 22791, National Bureau of Economic Research, Inc.
    23. Cheng Hsiao & H. Steve Ching & Shui Ki Wan, 2012. "A Panel Data Approach For Program Evaluation: Measuring The Benefits Of Political And Economic Integration Of Hong Kong With Mainland China," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(5), pages 705-740, August.
    24. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    25. Carvalho, Carlos & Masini, Ricardo & Medeiros, Marcelo C., 2018. "ArCo: An artificial counterfactual approach for high-dimensional panel time-series data," Journal of Econometrics, Elsevier, vol. 207(2), pages 352-380.
    26. Ferman, Bruno & Pinto, Cristine, 2017. "Placebo Tests for Synthetic Controls," MPRA Paper 78079, University Library of Munich, Germany.
    27. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    28. Hoff, Peter D., 2017. "Lasso, fractional norm and structured sparse estimation using a Hadamard product parametrization," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 186-198.
    29. Ian T. Jolliffe, 1982. "A Note on the Use of Principal Components in Regression," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 31(3), pages 300-303, November.
    30. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    31. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    32. King, Gary & Zeng, Langche, 2006. "The Dangers of Extreme Counterfactuals," Political Analysis, Cambridge University Press, vol. 14(2), pages 131-159, April.
    33. Alberto Abadie & Jérémy L’Hour, 2021. "A Penalized Synthetic Control Estimator for Disaggregated Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1817-1834, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    2. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    3. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2023. "Same Root Different Leaves: Time Series and Cross‐Sectional Methods in Panel Data," Econometrica, Econometric Society, vol. 91(6), pages 2125-2154, November.
    4. Luis Costa & Vivek F. Farias & Patricio Foncea & Jingyuan (Donna) Gan & Ayush Garg & Ivo Rosa Montenegro & Kumarjit Pathak & Tianyi Peng & Dusan Popovic, 2023. "Generalized Synthetic Control for TestOps at ABI: Models, Algorithms, and Infrastructure," Interfaces, INFORMS, vol. 53(5), pages 336-349, September.
    5. Zongwu Cai & Ying Fang & Ming Lin & Zixuan Wu, 2023. "A Quasi Synthetic Control Method for Nonlinear Models With High-Dimensional Covariates," WORKING PAPERS SERIES IN THEORETICAL AND APPLIED ECONOMICS 202305, University of Kansas, Department of Economics, revised Aug 2023.
    6. Viviano, Davide & Bradic, Jelena, 2023. "Synthetic Learner: Model-free inference on treatments over time," Journal of Econometrics, Elsevier, vol. 234(2), pages 691-713.
    7. Xingyu Li & Yan Shen & Qiankun Zhou, 2022. "Confidence Intervals of Treatment Effects in Panel Data Models with Interactive Fixed Effects," Papers 2202.12078, arXiv.org.
    8. Michael Funke & Kadri Männasoo & Helery Tasane, 2023. "Regional Economic Impacts of the Øresund Cross-Border Fixed Link: Cui Bono?," CESifo Working Paper Series 10557, CESifo.
    9. Roy Cerqueti & Raffaella Coppier & Alessandro Girardi & Marco Ventura, 2022. "The sooner the better: lives saved by the lockdown during the COVID-19 outbreak. The case of Italy [Using synthetic controls: Feasibility, data requirements, and methodological aspects]," The Econometrics Journal, Royal Economic Society, vol. 25(1), pages 46-70.
    10. Lea Bottmer & Guido Imbens & Jann Spiess & Merrill Warnick, 2021. "A Design-Based Perspective on Synthetic Control Methods," Papers 2101.09398, arXiv.org, revised Jul 2023.
    11. Alberto Abadie & Jaume Vives-i-Bastida, 2022. "Synthetic Controls in Action," Papers 2203.06279, arXiv.org.
    12. Joseph Fry, 2023. "A Method of Moments Approach to Asymptotically Unbiased Synthetic Controls," Papers 2312.01209, arXiv.org, revised Mar 2024.
    13. Victor Chernozhukov & Kaspar Wüthrich & Yinchu Zhu, 2019. "Inference on average treatment effects in aggregate panel data settings," CeMMAP working papers CWP32/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    14. Victor Chernozhukov & Kaspar Wuthrich & Yinchu Zhu, 2018. "A $t$-test for synthetic controls," Papers 1812.10820, arXiv.org, revised Jan 2024.
    15. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    16. Ignacio Martinez & Jaume Vives-i-Bastida, 2022. "Bayesian and Frequentist Inference for Synthetic Controls," Papers 2206.01779, arXiv.org, revised Feb 2023.
    17. Guillaume Allaire Pouliot & Zhen Xie, 2022. "Degrees of Freedom and Information Criteria for the Synthetic Control Method," Papers 2207.02943, arXiv.org.
    18. Masahiro Kato & Akari Ohda & Masaaki Imaizumi & Kenichiro McAlinn, 2023. "Synthetic Control Methods by Density Matching under Implicit Endogeneity," Papers 2307.11127, arXiv.org, revised Jul 2023.
    19. Anish Agarwal & Vasilis Syrgkanis, 2022. "Synthetic Blip Effects: Generalizing Synthetic Controls for the Dynamic Treatment Regime," Papers 2210.11003, arXiv.org.
    20. Peter Backus & Thien Nguyen, 2021. "The Effect of the Sex Buyer Law on the Market for Sex, Sexual Health and Sexual Violence," Economics Discussion Paper Series 2106, Economics, The University of Manchester.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2207.14481. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.