IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2202.02903.html
   My bibliography  Save this paper

Difference-in-Differences with Time-Varying Covariates in the Parallel Trends Assumption

Author

Listed:
  • Carolina Caetano
  • Brantly Callaway

Abstract

In this paper, we study difference-in-differences identification and estimation strategies where the parallel trends assumption holds after conditioning on time-varying covariates and/or time-invariant covariates. Our first main contribution is to point out a number of weaknesses of commonly used two-way fixed effects (TWFE) regressions in this context. In addition to issues related to multiple periods and variation in treatment timing that have been emphasized in the literature, we show that, even in the case with only two time periods, TWFE regressions are not generally robust to (i) paths of untreated potential outcomes depending on the level of time-varying covariates (as opposed to only the change in the covariates over time), (ii) paths of untreated potential outcomes depending on time-invariant covariates, and (iii) violations of linearity conditions for outcomes over time and/or the propensity score. Even in cases where none of the previous three issues hold, we show that TWFE regressions can suffer from negative weighting and weight-reversal issues. Thus, TWFE regressions can deliver misleading estimates of causal effect parameters in a number of empirically relevant cases. Second, we extend these arguments to the case of multiple periods and variation in treatment timing. Third, we provide simple diagnostics for assessing the extent of misspecification bias arising due to TWFE regressions. Finally, we propose alternative (and simple) estimation strategies that can circumvent these issues with two-way fixed regressions.

Suggested Citation

  • Carolina Caetano & Brantly Callaway, 2022. "Difference-in-Differences with Time-Varying Covariates in the Parallel Trends Assumption," Papers 2202.02903, arXiv.org, revised May 2023.
  • Handle: RePEc:arx:papers:2202.02903
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2202.02903
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    2. Barnette Justin & Reynolds C. Lockwood & Odongo Kennedy, 2021. "Changes Over Time in the Cost of Job Loss for Young Men and Women," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 21(1), pages 335-378, January.
    3. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    4. Topel, Robert H, 1991. "Specific Capital, Mobility, and Wages: Wages Rise with Job Seniority," Journal of Political Economy, University of Chicago Press, vol. 99(1), pages 145-176, February.
    5. Lechner, Michael, 2008. "A note on endogenous control variables in causal studies," Statistics & Probability Letters, Elsevier, vol. 78(2), pages 190-195, February.
    6. Shoya Ishimaru, 2021. "What Do We Get from Two-Way Fixed Effects Regressions? Implications from Numerical Equivalence," Papers 2103.12374, arXiv.org, revised Jan 2024.
    7. Peter M. Aronow & Cyrus Samii, 2016. "Does Regression Produce Representative Estimates of Causal Effects?," American Journal of Political Science, John Wiley & Sons, vol. 60(1), pages 250-267, January.
    8. Joakim A. Weill & Matthieu Stigler & Olivier Deschenes & Michael R. Springborn, 2021. "Researchers' Degrees-of-Flexibility and the Credibility of Difference-in-Differences Estimates: Evidence From the Pandemic Policy Evaluations," NBER Working Papers 29550, National Bureau of Economic Research, Inc.
    9. Blackwell, Matthew & Glynn, Adam N., 2018. "How to Make Causal Inferences with Time-Series Cross-Sectional Data under Selection on Observables," American Political Science Review, Cambridge University Press, vol. 112(4), pages 1067-1082, November.
    10. Callaway, Brantly & Li, Tong & Oka, Tatsushi, 2018. "Quantile treatment effects in difference in differences models under dependence restrictions and with only two time periods," Journal of Econometrics, Elsevier, vol. 206(2), pages 395-413.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nicholas Brown & Kyle Butts & Joakim Westerlund, 2023. "Simple Difference-in-Differences Estimation in Fixed-T Panels," Papers 2301.11358, arXiv.org, revised Jun 2023.
    2. Maclean, Johanna Catherine & Tello-Trillo, Sebastian & Webber, Douglas, 2023. "Losing insurance and psychiatric hospitalizations," Journal of Economic Behavior & Organization, Elsevier, vol. 205(C), pages 508-527.
    3. Dalia Ghanem & Pedro H. C. Sant'Anna & Kaspar Wüthrich, 2022. "Selection and Parallel Trends," CESifo Working Paper Series 9910, CESifo.
    4. Luis Antonio Fantozzi Alvarez & Rodrigo Toneto, 2024. "The interpretation of 2SLS with a continuous instrument: a weighted LATE representation," Working Papers, Department of Economics 2024_11, University of São Paulo (FEA-USP).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Brantly Callaway, 2022. "Difference-in-Differences for Policy Evaluation," Papers 2203.15646, arXiv.org.
    2. Emanuela Galasso & Martin Ravallion, 2004. "Social Protection in a Crisis: Argentina's Plan Jefes y Jefas," The World Bank Economic Review, World Bank, vol. 18(3), pages 367-399.
    3. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    4. Augurzky, Boris & Kluve, Jochen, 2004. "Assessing the performance of matching algorithms when selection into treatment is strong," RWI Discussion Papers 21, RWI - Leibniz-Institut für Wirtschaftsforschung.
    5. Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
    6. Frölich, Markus & Michaelowa, Katharina, 2004. "Peer effects and textbooks in primary education: Evidence from francophone sub-Saharan Africa," HWWA Discussion Papers 311, Hamburg Institute of International Economics (HWWA).
    7. Malcolm Keswell & Michael R. Carter, 2011. "Poverty and Land Distribution: Evidence from a Natural Experiment," WIDER Working Paper Series wp-2011-046, World Institute for Development Economic Research (UNU-WIDER).
    8. Sloczynski, Tymon, 2020. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," IZA Discussion Papers 13283, Institute of Labor Economics (IZA).
    9. Kluve, Jochen & Lehmann, Hartmut & Schmidt, Christoph M., 1999. "Active Labor Market Policies in Poland: Human Capital Enhancement, Stigmatization, or Benefit Churning?," Journal of Comparative Economics, Elsevier, vol. 27(1), pages 61-89, March.
    10. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2006. "Moving the Goalposts: Addressing Limited Overlap in the Estimation of Average Treatment Effects by Changing the Estimand," NBER Technical Working Papers 0330, National Bureau of Economic Research, Inc.
    11. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    12. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    13. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    14. Markus Frölich, 2004. "Programme Evaluation with Multiple Treatments," Journal of Economic Surveys, Wiley Blackwell, vol. 18(2), pages 181-224, April.
    15. Schmid, Thomas & Ampenberger, Markus & Kaserer, Christoph & Achleitner, Ann-Kristin, 2010. "Controlling shareholders and payout policy: do founding families have a special 'taste for dividends'?," CEFS Working Paper Series 2010-01, Technische Universität München (TUM), Center for Entrepreneurial and Financial Studies (CEFS).
    16. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    17. Jochen Kluve & Boris Augurzky, 2007. "Assessing the performance of matching algorithms when selection into treatment is strong," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(3), pages 533-557.
    18. Czarnitzki, Dirk & Lopes-Bento, Cindy, 2013. "Value for money? New microeconometric evidence on public R&D grants in Flanders," Research Policy, Elsevier, vol. 42(1), pages 76-89.
    19. Alm Bastian & Engel Dirk & Weyh Antje, 2014. "Does Switching to a Western German Employer Still Pay Off?: An Analysis for Eastern Germany," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 234(5), pages 546-571, October.
    20. Hottenrott, Hanna & Lopes-Bento, Cindy, 2014. "(International) R&D collaboration and SMEs: The effectiveness of targeted public R&D support schemes," Research Policy, Elsevier, vol. 43(6), pages 1055-1066.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2202.02903. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.