IDEAS home Printed from https://ideas.repec.org/a/oup/emjrnl/v26y2023i1p1-24..html
   My bibliography  Save this article

Combining counterfactual outcomes and ARIMA models for policy evaluation

Author

Listed:
  • Fiammetta Menchetti
  • Fabrizio Cipollini
  • Fabrizia Mealli

Abstract

SummaryThe Rubin Causal Model (RCM) is a framework that allows to define the causal effect of an intervention as a contrast of potential outcomes. In recent years, several methods have been developed under the RCM to estimate causal effects in time series settings. None of these makes use of autoregressive integrated moving average (ARIMA) models, which are instead very common in the econometrics literature. In this paper, we propose a novel approach, named Causal-ARIMA (C-ARIMA), to define and estimate the causal effect of an intervention in observational time series settings under the RCM. We first formalise the assumptions enabling the definition, the estimation and the attribution of the effect to the intervention. We then check the validity of the proposed method with a simulation study. In the empirical application, we use C-ARIMA to assess the causal effect of a permanent price reduction on supermarket sales. The CausalArima R package provides an implementation of the proposed approach.

Suggested Citation

  • Fiammetta Menchetti & Fabrizio Cipollini & Fabrizia Mealli, 2023. "Combining counterfactual outcomes and ARIMA models for policy evaluation," The Econometrics Journal, Royal Economic Society, vol. 26(1), pages 1-24.
  • Handle: RePEc:oup:emjrnl:v:26:y:2023:i:1:p:1-24.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/ectj/utac024
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Eli Ben-Michael & Avi Feller & Jesse Rothstein, 2021. "The Augmented Synthetic Control Method," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 116(536), pages 1789-1803, October.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. repec:fth:prinin:315 is not listed on IDEAS
    4. Iavor Bojinov & Neil Shephard, 2019. "Time Series Experiments and Causal Estimands: Exact Randomization Tests and Trading," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 114(528), pages 1665-1682, October.
    5. Anger, Silke & Kvasnicka, Michael & Siedler, Thomas, 2011. "One Last Puff? Public Smoking Bans and Smoking Behavior," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 30(3), pages 591-601.
    6. Larcker, David F. & Gordon, Lawrence A. & Pinches, George E., 1980. "Testing for Market Efficiency: A Comparison of the Cumulative Average Residual Methodology and Intervention Analysis," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 15(2), pages 267-287, June.
    7. Andrew Worthington & Abbas Valadkhani, 2004. "Measuring the impact of natural disasters on capital markets: an empirical application using intervention analysis," Applied Economics, Taylor & Francis Journals, vol. 36(19), pages 2177-2186.
    8. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    9. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    10. Irene Botosaru & Bruno Ferman, 2019. "On the role of covariates in the synthetic control method," The Econometrics Journal, Royal Economic Society, vol. 22(2), pages 117-130.
    11. Andreas Billmeier & Tommaso Nannicini, 2013. "Assessing Economic Liberalization Episodes: A Synthetic Control Approach," The Review of Economics and Statistics, MIT Press, vol. 95(3), pages 983-1001, July.
    12. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    13. David Card & Alan Krueger, 1993. "Minimum Wages and Employment: A Case Study of the Fast Food Industry in New Jersey and Pennsylvania," Working Papers 694, Princeton University, Department of Economics, Industrial Relations Section..
    14. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    15. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    16. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    17. Sims, Christopher A, 1972. "Money, Income, and Causality," American Economic Review, American Economic Association, vol. 62(4), pages 540-552, September.
    18. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    19. Michael Lechner, 2011. "The Relation of Different Concepts of Causality Used in Time Series and Microeconometrics," Econometric Reviews, Taylor & Francis Journals, vol. 30(1), pages 109-127.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fiaschi, Davide & Tealdi, Cristina, 2024. "Let's Roll Back! The Challenging Task of Regulating Temporary Contracts," IZA Discussion Papers 16777, Institute of Labor Economics (IZA).
    2. Inkyu Kang, 2023. "How does technology‐based monitoring affect street‐level bureaucrats' behavior? An analysis of body‐worn cameras and police actions," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(4), pages 971-991, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    2. Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org, revised Aug 2022.
    3. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.
    4. Viviano, Davide & Bradic, Jelena, 2023. "Synthetic Learner: Model-free inference on treatments over time," Journal of Econometrics, Elsevier, vol. 234(2), pages 691-713.
    5. Michael Funke & Kadri Männasoo & Helery Tasane, 2023. "Regional Economic Impacts of the Øresund Cross-Border Fixed Link: Cui Bono?," CESifo Working Paper Series 10557, CESifo.
    6. Eli Ben‐Michael & Avi Feller & Jesse Rothstein, 2022. "Synthetic controls with staggered adoption," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 351-381, April.
    7. Roy Cerqueti & Raffaella Coppier & Alessandro Girardi & Marco Ventura, 2022. "The sooner the better: lives saved by the lockdown during the COVID-19 outbreak. The case of Italy [Using synthetic controls: Feasibility, data requirements, and methodological aspects]," The Econometrics Journal, Royal Economic Society, vol. 25(1), pages 46-70.
    8. Lea Bottmer & Guido Imbens & Jann Spiess & Merrill Warnick, 2021. "A Design-Based Perspective on Synthetic Control Methods," Papers 2101.09398, arXiv.org, revised Jul 2023.
    9. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    10. Juan S. Mora‐Sanguinetti & Rok Spruk, 2023. "Economic effects of recent experiences of federalism: Analysis of the regionalization process in Spain," Journal of Regional Science, Wiley Blackwell, vol. 63(1), pages 30-63, January.
    11. Joakim Weill, 2023. "Flood Risk Mapping and the Distributional Impacts of Climate Information," Working Papers 2023.10, FAERE - French Association of Environmental and Resource Economists.
    12. Emery, Thomas & Mélon, Lela & Spruk, Rok, 2023. "Does e-procurement matter for economic growth? Subnational evidence from Australia," The Quarterly Review of Economics and Finance, Elsevier, vol. 89(C), pages 318-334.
    13. Joakim A. Weill, 2023. "Flood Risk Mapping and the Distributional Impacts of Climate Information," Finance and Economics Discussion Series 2023-066, Board of Governors of the Federal Reserve System (U.S.).
    14. David Gilchrist & Thomas Emery & Nuno Garoupa & Rok Spruk, 2023. "Synthetic Control Method: A tool for comparative case studies in economic history," Journal of Economic Surveys, Wiley Blackwell, vol. 37(2), pages 409-445, April.
    15. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    16. Davide Viviano & Jelena Bradic, 2021. "Dynamic covariate balancing: estimating treatment effects over time with potential local projections," Papers 2103.01280, arXiv.org, revised Jan 2024.
    17. Dmitry Arkhangelsky & Vasily Korovkin, 2020. "On Policy Evaluation with Aggregate Time-Series Shocks," CERGE-EI Working Papers wp657, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    18. Bruno Ferman & Cristine Pinto & Vitor Possebom, 2020. "Cherry Picking with Synthetic Controls," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 39(2), pages 510-532, March.
    19. Maximiliano Marzetti & Rok Spruk, 2023. "Long-Term Economic Effects of Populist Legal Reforms: Evidence from Argentina," Comparative Economic Studies, Palgrave Macmillan;Association for Comparative Economic Studies, vol. 65(1), pages 60-95, March.
    20. Kirill Borusyak & Xavier Jaravel & Jann Spiess, 2021. "Revisiting Event Study Designs: Robust and Efficient Estimation," Papers 2108.12419, arXiv.org, revised Jan 2024.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:emjrnl:v:26:y:2023:i:1:p:1-24.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://edirc.repec.org/data/resssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.