IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2503.13323.html
   My bibliography  Save this paper

Difference-in-Differences Designs: A Practitioner's Guide

Author

Listed:
  • Andrew Baker
  • Brantly Callaway
  • Scott Cunningham
  • Andrew Goodman-Bacon
  • Pedro H. C. Sant'Anna

Abstract

Difference-in-Differences (DiD) is arguably the most popular quasi-experimental research design. Its canonical form, with two groups and two periods, is well-understood. However, empirical practices can be ad hoc when researchers go beyond that simple case. This article provides an organizing framework for discussing different types of DiD designs and their associated DiD estimators. It discusses covariates, weights, handling multiple periods, and staggered treatments. The organizational framework, however, applies to other extensions of DiD methods as well.

Suggested Citation

  • Andrew Baker & Brantly Callaway & Scott Cunningham & Andrew Goodman-Bacon & Pedro H. C. Sant'Anna, 2025. "Difference-in-Differences Designs: A Practitioner's Guide," Papers 2503.13323, arXiv.org.
    • Handle: RePEc:arx:papers:2503.13323
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2503.13323
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Iván Fernández‐Val & Ye Luo, 2018. "The Sorted Effects Method: Discovering Heterogeneous Effects Beyond Their Averages," Econometrica, Econometric Society, vol. 86(6), pages 1911-1938, November.
    2. Meyer, Bruce D & Viscusi, W Kip & Durbin, David L, 1995. "Workers' Compensation and Injury Duration: Evidence from a Natural Experiment," American Economic Review, American Economic Association, vol. 85(3), pages 322-340, June.
    3. Fan, Yanqin & Yu, Zhengfei, 2012. "Partial identification of distributional and quantile treatment effects in difference-in-differences models," Economics Letters, Elsevier, vol. 115(3), pages 511-515.
    4. Borgschulte, Mark & Vogler, Jacob, 2020. "Did the ACA Medicaid expansion save lives?," Journal of Health Economics, Elsevier, vol. 72(C).
    5. Janet Currie & Hannes Schwandt, 2016. "Mortality Inequality: The Good News from a County-Level Approach," Journal of Economic Perspectives, American Economic Association, vol. 30(2), pages 29-52, Spring.
    6. Black, Bernard & Hollingsworth, Alex & Nunes, Letícia & Simon, Kosali, 2022. "Simulated power analyses for observational studies: An application to the Affordable Care Act Medicaid expansion," Journal of Public Economics, Elsevier, vol. 213(C).
    7. Victor Chernozhukov & Mert Demirer & Esther Duflo & Iv'an Fern'andez-Val, 2017. "Fisher-Schultz Lecture: Generic Machine Learning Inference on Heterogenous Treatment Effects in Randomized Experiments, with an Application to Immunization in India," Papers 1712.04802, arXiv.org, revised Oct 2023.
    8. Guido Imbens & Nathan Kallus & Xiaojie Mao, 2021. "Controlling for Unmeasured Confounding in Panel Data Using Minimal Bridge Functions: From Two-Way Fixed Effects to Factor Models," Papers 2108.03849, arXiv.org.
    9. Kirill Borusyak & Xavier Jaravel & Jann Spiess, 2024. "Revisiting Event-Study Designs: Robust and Efficient Estimation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(6), pages 3253-3285.
    10. Sasaki, Yuya & Ura, Takuya, 2022. "Estimation And Inference For Moments Of Ratios With Robustness Against Large Trimming Bias," Econometric Theory, Cambridge University Press, vol. 38(1), pages 66-112, February.
    11. Neng-Chieh Chang, 2020. "Double/debiased machine learning for difference-in-differences models," The Econometrics Journal, Royal Economic Society, vol. 23(2), pages 177-191.
    12. Paul Goldsmith-Pinkham & Peter Hull & Michal Kolesár, 2022. "Contamination Bias in Linear Regressions," Working Papers 2022-15, Princeton University. Economics Department..
    13. Alberto Abadie & Susan Athey & Guido W. Imbens & Jeffrey M. Wooldridge, 2020. "Sampling‐Based versus Design‐Based Uncertainty in Regression Analysis," Econometrica, Econometric Society, vol. 88(1), pages 265-296, January.
    14. Paul Goldsmith-Pinkham & Peter Hull & Michal Kolesár, 2024. "Contamination Bias in Linear Regressions," American Economic Review, American Economic Association, vol. 114(12), pages 4015-4051, December.
    15. Manasi Deshpande & Yue Li, 2019. "Who Is Screened Out? Application Costs and the Targeting of Disability Programs," American Economic Journal: Economic Policy, American Economic Association, vol. 11(4), pages 213-248, November.
    16. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 487-535.
    17. Ricardo Mora & Iliana Reggio, 2019. "Alternative diff-in-diffs estimators with several pretreatment periods," Econometric Reviews, Taylor & Francis Journals, vol. 38(5), pages 465-486, May.
    18. Ashesh Rambachan & Jonathan Roth, 2023. "A More Credible Approach to Parallel Trends," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(5), pages 2555-2591.
    19. Malani, Anup & Reif, Julian, 2015. "Interpreting pre-trends as anticipation: Impact on estimated treatment effects from tort reform," Journal of Public Economics, Elsevier, vol. 124(C), pages 1-17.
    20. Carolina Caetano & Brantly Callaway, 2024. "Difference-in-Differences when Parallel Trends Holds Conditional on Covariates," Papers 2406.15288, arXiv.org, revised Sep 2024.
    21. Janet Currie & Henrik Kleven & Esmée Zwiers, 2020. "Technology and Big Data Are Changing Economics: Mining Text to Track Methods," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 42-48, May.
    22. Joshua D. Angrist, 1998. "Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants," Econometrica, Econometric Society, vol. 66(2), pages 249-288, March.
    23. Bryan S. Graham & Cristine Campos De Xavier Pinto & Daniel Egel, 2012. "Inverse Probability Tilting for Moment Condition Models with Missing Data," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 1053-1079.
    24. Alexandre Poirier & Tymon S{l}oczy'nski, 2024. "Quantifying the Internal Validity of Weighted Estimands," Papers 2404.14603, arXiv.org, revised Mar 2025.
    25. Itzik Fadlon & Torben Heien Nielsen, 2021. "Family Labor Supply Responses to Severe Health Shocks: Evidence from Danish Administrative Records," American Economic Journal: Applied Economics, American Economic Association, vol. 13(3), pages 1-30, July.
    26. Stephen G. Donald & Kevin Lang, 2007. "Inference with Difference-in-Differences and Other Panel Data," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 221-233, May.
    27. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    28. Charles F. Manski & John V. Pepper, 2018. "How Do Right-to-Carry Laws Affect Crime Rates? Coping with Ambiguity Using Bounded-Variation Assumptions," The Review of Economics and Statistics, MIT Press, vol. 100(2), pages 232-244, May.
    29. Stéphane Bonhomme & Ulrich Sauder, 2011. "Recovering Distributions in Difference-in-Differences Models: A Comparison of Selective and Comprehensive Schooling," The Review of Economics and Statistics, MIT Press, vol. 93(2), pages 479-494, May.
    30. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    31. Finkelstein, Amy & McKnight, Robin, 2008. "What did Medicare do? The initial impact of Medicare on mortality and out of pocket medical spending," Journal of Public Economics, Elsevier, vol. 92(7), pages 1644-1668, July.
    32. Jaap H. Abbring & Gerard J. van den Berg, 2003. "The Nonparametric Identification of Treatment Effects in Duration Models," Econometrica, Econometric Society, vol. 71(5), pages 1491-1517, September.
    33. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, December.
    34. Holger Dette & Martin Schumann, 2024. "Testing for Equivalence of Pre-Trends in Difference-in-Differences Estimation," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(4), pages 1289-1301, October.
    35. Yukun Ma & Pedro H. C. Sant'Anna & Yuya Sasaki & Takuya Ura, 2023. "Doubly Robust Estimators with Weak Overlap," Papers 2304.08974, arXiv.org, revised Apr 2023.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    2. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    3. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    4. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    5. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    6. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    7. Gregory Faletto, 2023. "Fused Extended Two-Way Fixed Effects for Difference-in-Differences With Staggered Adoptions," Papers 2312.05985, arXiv.org, revised Apr 2025.
    8. Callaway, Brantly & Karami, Sonia, 2023. "Treatment effects in interactive fixed effects models with a small number of time periods," Journal of Econometrics, Elsevier, vol. 233(1), pages 184-208.
    9. Alexandre Poirier & Tymon S{l}oczy'nski, 2024. "Quantifying the Internal Validity of Weighted Estimands," Papers 2404.14603, arXiv.org, revised Mar 2025.
    10. Kotyrlo, Elena, 2024. "Simple and complex difference-in-differences approach," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 73, pages 119-142.
    11. Ben Deaner & Chen-Wei Hsiang & Andrei Zeleneev, 2025. "Inferring Treatment Effects in Large Panels by Uncovering Latent Similarities," Papers 2503.20769, arXiv.org, revised Mar 2025.
    12. Kirill Borusyak & Xavier Jaravel & Jann Spiess, 2024. "Revisiting Event-Study Designs: Robust and Efficient Estimation," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 91(6), pages 3253-3285.
    13. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    14. Jiafeng Chen, 2021. "Nonparametric Treatment Effect Identification in School Choice," Papers 2112.03872, arXiv.org, revised Oct 2023.
    15. Lechner, Michael, 2011. "The Estimation of Causal Effects by Difference-in-Difference Methods," Foundations and Trends(R) in Econometrics, now publishers, vol. 4(3), pages 165-224, November.
    16. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    17. Shinde, Nilesh N. & Do Valle, Stella Z. Schons & Maia, Alexandre Gori & Amacher, Gregory S., 2022. "Can an environmental policy contribute to the reduction of land conflict? Evidence from the Rural Environmental Registry (CAR) in the Brazilian Amazon," 2022 Annual Meeting, July 31-August 2, Anaheim, California 322584, Agricultural and Applied Economics Association.
    18. Nikolay Doudchenko & Guido W. Imbens, 2016. "Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis," NBER Working Papers 22791, National Bureau of Economic Research, Inc.
    19. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    20. Arne Henningsen & Guy Low & David Wuepper & Tobias Dalhaus & Hugo Storm & Dagim Belay & Stefan Hirsch, 2024. "Estimating Causal Effects with Observational Data: Guidelines for Agricultural and Applied Economists," IFRO Working Paper 2024/03, University of Copenhagen, Department of Food and Resource Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2503.13323. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.