IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2106.10949.html
   My bibliography  Save this paper

On the Use of Two-Way Fixed Effects Models for Policy Evaluation During Pandemics

Author

Listed:
  • Germain Gauthier

Abstract

In the context of the Covid-19 pandemic, multiple studies rely on two-way fixed effects (FE) models to assess the impact of mitigation policies on health outcomes. Building on the SIRD model of disease transmission, I show that FE models tend to be misspecified for three reasons. First, despite misleading common trends in the pre-treatment period, the parallel trends assumption generally does not hold. Second, heterogeneity in infection rates and infected populations across regions cannot be accounted for by region-specific fixed effects, nor by conditioning on observable time-varying confounders. Third, epidemiological theory predicts heterogeneous treatment effects across regions and over time. Via simulations, I find that the bias resulting from model misspecification can be substantial, in magnitude and sometimes in sign. Overall, my results caution against the use of FE models for mitigation policy evaluation.

Suggested Citation

  • Germain Gauthier, 2021. "On the Use of Two-Way Fixed Effects Models for Policy Evaluation During Pandemics," Papers 2106.10949, arXiv.org.
  • Handle: RePEc:arx:papers:2106.10949
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2106.10949
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. James H. Fowler & Seth J. Hill & Remy Levin & Nick Obradovich, 2020. "The effect of stay-at-home orders on COVID-19 cases and fatalities in the United States," Papers 2004.06098, arXiv.org, revised May 2020.
    2. Steffen Juranek & Floris Zoutman, 2020. "The Effect of Social Distancing Measures on the Demand for Intensive Care: Evidence on Covid-19 in Scandinavia," CESifo Working Paper Series 8262, CESifo.
    3. Karaivanov, Alexander & Lu, Shih En & Shigeoka, Hitoshi & Chen, Cong & Pamplona, Stephanie, 2021. "Face masks, public policies and slowing the spread of COVID-19: Evidence from Canada," Journal of Health Economics, Elsevier, vol. 78(C).
    4. Persico, Claudia L. & Johnson, Kathryn R., 2021. "The effects of increased pollution on COVID-19 cases and deaths," Journal of Environmental Economics and Management, Elsevier, vol. 107(C).
    5. Juranek, Steffen & Zoutman, Floris T., 2020. "The Effect of Social Distancing Measures on Intensive Care Occupancy: Evidence on COVID-19 in Scandinavia," Discussion Papers 2020/2, Norwegian School of Economics, Department of Business and Management Science.
    6. Mitze, Timo & Kosfeld, Reinhold & Rode, Johannes & Wälde, Klaus, 2020. "Face masks considerably reduce COVID-19 cases in Germany," Publications of Darmstadt Technical University, Institute for Business Studies (BWL) 124587, Darmstadt Technical University, Department of Business Administration, Economics and Law, Institute for Business Studies (BWL).
    7. Dhaval M. Dave & Andrew I. Friedson & Kyutaro Matsuzawa & Joseph J. Sabia & Samuel Safford, 2020. "Were Urban Cowboys Enough to Control COVID-19? Local Shelter-in-Place Orders and Coronavirus Case Growth," NBER Working Papers 27229, National Bureau of Economic Research, Inc.
    8. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    9. Sang-Wook (Stanley) Cho, 2020. "Quantifying the impact of nonpharmaceutical interventions during the COVID-19 outbreak: The case of Sweden," Econometrics Journal, Royal Economic Society, vol. 23(3), pages 323-344.
    10. Dhaval Dave & Andrew I. Friedson & Kyutaro Matsuzawa & Joseph J. Sabia, 2021. "When Do Shelter‐In‐Place Orders Fight Covid‐19 Best? Policy Heterogeneity Across States And Adoption Time," Economic Inquiry, Western Economic Association International, vol. 59(1), pages 29-52, January.
    11. Friedson, Andrew I. & McNichols, Drew & Sabia, Joseph J. & Dave, Dhaval M., 2020. "Did California's Shelter-In-Place Order Work? Early Coronavirus-Related Public Health Effects," IZA Discussion Papers 13160, Institute of Labor Economics (IZA).
    12. Bursztyn, Leonardo & Rao, Akaash & Roth, Christopher & Yanagizawa-Drott, David, 2020. "Misinformation during a Pandemic," The Warwick Economics Research Paper Series (TWERPS) 1274, University of Warwick, Department of Economics.
    13. Andrew Goodman-Bacon, 2018. "Difference-in-Differences with Variation in Treatment Timing," NBER Working Papers 25018, National Bureau of Economic Research, Inc.
    14. Chernozhukov, Victor & Kasahara, Hiroyuki & Schrimpf, Paul, 2021. "Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S," Journal of Econometrics, Elsevier, vol. 220(1), pages 23-62.
    15. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    16. Goodman-Bacon, Andrew & Marcus, Jan, 2020. "Using Difference-in-Differences to Identify Causal Effects of COVID-19 Policies," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, pages 153-158.
    17. Ash, Elliott & Galletta, Sergio & Hangartner, Dominik & Margalit, Yotam & Pinna, Matteo, 2020. "The Effect of Fox News on Health Behavior During COVID-19," SocArXiv abqe5, Center for Open Science.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brantly Callaway & Tong Li, 2021. "Policy Evaluation during a Pandemic," Papers 2105.06927, arXiv.org.
    2. Léa BOU SLEIMAN & Germain GAUTHIER, 2020. "COVID-19: Reduced forms have gone viral, but what do they tell us?," Working Papers 2020-32, Center for Research in Economics and Statistics, revised 18 Jan 2021.
    3. Wright, Austin L. & Sonin, Konstantin & Driscoll, Jesse & Wilson, Jarnickae, 2020. "Poverty and economic dislocation reduce compliance with COVID-19 shelter-in-place protocols," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 544-554.
    4. Catalina Amuedo-Dorantes & Neeraj Kaushal & Ashley N. Muchow, 2021. "Timing of social distancing policies and COVID-19 mortality: county-level evidence from the U.S," Journal of Population Economics, Springer;European Society for Population Economics, vol. 34(4), pages 1445-1472, October.
    5. Karaivanov, Alexander & Lu, Shih En & Shigeoka, Hitoshi & Chen, Cong & Pamplona, Stephanie, 2021. "Face masks, public policies and slowing the spread of COVID-19: Evidence from Canada," Journal of Health Economics, Elsevier, vol. 78(C).
    6. Edoardo Di Porto & Paolo Naticchioni & Vincenzo Scrutinio, 2020. "Partial Lockdown and the Spread of Covid-19: Lessons from the Italian Case," CSEF Working Papers 569, Centre for Studies in Economics and Finance (CSEF), University of Naples, Italy.
    7. Martin Huber & Henrika Langen, 2020. "Timing matters: the impact of response measures on COVID-19-related hospitalization and death rates in Germany and Switzerland," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 156(1), pages 1-19, December.
    8. Carlos B. Carneiro & I'uri H. Ferreira & Marcelo C. Medeiros & Henrique F. Pires & Eduardo Zilberman, 2020. "Lockdown effects in US states: an artificial counterfactual approach," Papers 2009.13484, arXiv.org, revised Feb 2021.
    9. Andrew I. Friedson & Drew McNichols & Joseph J. Sabia & Dhaval Dave, 2021. "Shelter‐In‐Place Orders And Public Health: Evidence From California During The Covid‐19 Pandemic," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 40(1), pages 258-283, January.
    10. Dongwoo Kim & Young Jun Lee, 2021. "Vaccination strategies and transmission of COVID-19: evidence across leading countries," Papers 2109.06453, arXiv.org.
    11. Bruno Ferman, 2020. "Inference in Difference-in-Differences with Few Treated Units and Spatial Correlation," Papers 2006.16997, arXiv.org, revised Sep 2021.
    12. Robert J. R. Elliott & Ingmar Schumacher & Cees Withagen, 2020. "Suggestions for a Covid-19 Post-Pandemic Research Agenda in Environmental Economics," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 76(4), pages 1187-1213, August.
    13. Dave, Dhaval M. & Friedson, Andrew I. & Matsuzawa, Kyutaro & Sabia, Joseph J. & Safford, Samuel, 2020. "Black Lives Matter Protests, Social Distancing, and COVID-19," IZA Discussion Papers 13388, Institute of Labor Economics (IZA).
    14. Goolsbee, Austan & Syverson, Chad, 2021. "Fear, lockdown, and diversion: Comparing drivers of pandemic economic decline 2020," Journal of Public Economics, Elsevier, vol. 193(C).
    15. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    16. Steffen Juranek & Floris T. Zoutman, 2021. "The effect of non-pharmaceutical interventions on the demand for health care and on mortality: evidence from COVID-19 in Scandinavia," Journal of Population Economics, Springer;European Society for Population Economics, vol. 34(4), pages 1299-1320, October.
    17. Sumedha Gupta & Kosali I. Simon & Coady Wing, 2020. "Mandated and Voluntary Social Distancing During The COVID-19 Epidemic: A Review," NBER Working Papers 28139, National Bureau of Economic Research, Inc.
    18. Rösner, Anja & Haucap, Justus & Heimeshoff, Ulrich, 2020. "The impact of consumer protection in the digital age: Evidence from the European Union," International Journal of Industrial Organization, Elsevier, vol. 73(C).
    19. Bardey, David & Fernandez Sierra, Manuel & Gravel, Alexis, 2021. "Coronavirus and Social Distancing: Do Non-Pharmaceutical-Interventions Work (at Least) in the Short Run?," IZA Discussion Papers 14095, Institute of Labor Economics (IZA).
    20. Sumit Agarwal & Slava Mikhed & Barry Scholnick & Man Zhang, 2021. "Reducing Strategic Default in a Financial Crisis," Working Papers 21-36, Federal Reserve Bank of Philadelphia.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2106.10949. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.