IDEAS home Printed from https://ideas.repec.org/a/wly/japmet/v40y2025i3p291-309.html
   My bibliography  Save this article

Standard Errors for Difference‐in‐Difference Regression

Author

Listed:
  • Bruce E. Hansen

Abstract

This paper makes a case for the use of jackknife methods for standard error, p$$ p $$ value, and confidence interval construction for difference‐in‐difference (DiD) regression. We review cluster‐robust, bootstrap, and jackknife standard error methods and show that standard methods can substantially underperform in conventional settings. In contrast, our proposed jackknife inference methods work well in broad contexts. We illustrate the relevance by replicating several influential DiD applications and showing how inferential results can change if jackknife standard error and inference methods are used.

Suggested Citation

  • Bruce E. Hansen, 2025. "Standard Errors for Difference‐in‐Difference Regression," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 40(3), pages 291-309, April.
  • Handle: RePEc:wly:japmet:v:40:y:2025:i:3:p:291-309
    DOI: 10.1002/jae.3110
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/jae.3110
    Download Restriction: no

    File URL: https://libkey.io/10.1002/jae.3110?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    2. Guido W. Imbens & Michal Kolesár, 2016. "Robust Standard Errors in Small Samples: Some Practical Advice," The Review of Economics and Statistics, MIT Press, vol. 98(4), pages 701-712, October.
    3. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    4. Djogbenou, Antoine A. & MacKinnon, James G. & Nielsen, Morten Ørregaard, 2019. "Asymptotic theory and wild bootstrap inference with clustered errors," Journal of Econometrics, Elsevier, vol. 212(2), pages 393-412.
    5. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    6. Gianmaria Niccodemi & Tom Wansbeek, 2022. "A New Estimator for Standard Errors with Few Unbalanced Clusters," Econometrics, MDPI, vol. 10(1), pages 1-7, January.
    7. A. Colin Cameron & Jonah B. Gelbach & Douglas L. Miller, 2008. "Bootstrap-Based Improvements for Inference with Clustered Errors," The Review of Economics and Statistics, MIT Press, vol. 90(3), pages 414-427, August.
    8. Matias D. Cattaneo & Michael Jansson & Whitney K. Newey, 2018. "Inference in Linear Regression Models with Many Covariates and Heteroscedasticity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1350-1361, July.
    9. Slawa Rokicki & Jessica Cohen & Gunther Fink & Joshua Salomon & Mary Beth Landrum, 2018. "Inference with difference-in-differences with a small number of groups: a review, simulation study and empirical application using SHARE data," CHaRMS Working Papers 18-01, Centre for HeAlth Research at the Management School (CHaRMS).
    10. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Fast and reliable jackknife and bootstrap methods for cluster‐robust inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 671-694, August.
    11. Rustam Ibragimov & Ulrich K. Müller, 2016. "Inference with Few Heterogeneous Clusters," The Review of Economics and Statistics, MIT Press, vol. 98(1), pages 83-96, March.
    12. MacKinnon, James G., 2023. "Fast cluster bootstrap methods for linear regression models," Econometrics and Statistics, Elsevier, vol. 26(C), pages 52-71.
    13. Young, Alwyn, 2019. "Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results," LSE Research Online Documents on Economics 101401, London School of Economics and Political Science, LSE Library.
    14. Gautam Rao, 2019. "Familiarity Does Not Breed Contempt: Generosity, Discrimination, and Diversity in Delhi Schools," American Economic Review, American Economic Association, vol. 109(3), pages 774-809, March.
    15. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    16. Chesher, Andrew & Austin, Gerard, 1991. "The finite-sample distributions of heteroskedasticity robust Wald statistics," Journal of Econometrics, Elsevier, vol. 47(1), pages 153-173, January.
    17. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Leverage, influence, and the jackknife in clustered regression models: Reliable inference using summclust," Stata Journal, StataCorp LLC, vol. 23(4), pages 942-982, December.
    18. James E. Pustejovsky & Elizabeth Tipton, 2018. "Small-Sample Methods for Cluster-Robust Variance Estimation and Hypothesis Testing in Fixed Effects Models," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 36(4), pages 672-683, October.
    19. Martha J. Bailey, 2010. ""Momma's Got the Pill": How Anthony Comstock and Griswold v. Connecticut Shaped US Childbearing," American Economic Review, American Economic Association, vol. 100(1), pages 98-129, March.
    20. Chesher, Andrew, 1989. "Hajek Inequalities, Measures of Leverage and the Size of Heteroskedasticity Robust Wald Tests," Econometrica, Econometric Society, vol. 57(4), pages 971-977, July.
    21. Hagemann, Andreas, 2019. "Placebo inference on treatment effects when the number of clusters is small," Journal of Econometrics, Elsevier, vol. 213(1), pages 190-209.
    22. Arellano, M, 1987. "Computing Robust Standard Errors for Within-Groups Estimators," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 49(4), pages 431-434, November.
    23. MacKinnon, James G. & White, Halbert, 1985. "Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties," Journal of Econometrics, Elsevier, vol. 29(3), pages 305-325, September.
    24. Alwyn Young, 2019. "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 557-598.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    2. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Testing for the appropriate level of clustering in linear regression models," Journal of Econometrics, Elsevier, vol. 235(2), pages 2027-2056.
    3. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Fast and reliable jackknife and bootstrap methods for cluster‐robust inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(5), pages 671-694, August.
    4. Wang, Wenjie & Zhang, Yichong, 2024. "Wild bootstrap inference for instrumental variables regressions with weak and few clusters," Journal of Econometrics, Elsevier, vol. 241(1).
    5. James G. MacKinnon & Morten Ørregaard Nielsen & Matthew D. Webb, 2023. "Leverage, influence, and the jackknife in clustered regression models: Reliable inference using summclust," Stata Journal, StataCorp LLC, vol. 23(4), pages 942-982, December.
    6. James G. MacKinnon & Matthew D. Webb, 2020. "When and How to Deal with Clustered Errors in Regression Models," Working Paper 1421, Economics Department, Queen's University.
    7. Matthew D. Webb, 2023. "Reworking wild bootstrap‐based inference for clustered errors," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 56(3), pages 839-858, August.
    8. Jeffrey D. Michler & Anna Josephson, 2022. "Recent developments in inference: practicalities for applied economics," Chapters, in: A Modern Guide to Food Economics, chapter 11, pages 235-268, Edward Elgar Publishing.
    9. Kranz, Sebastian, 2024. "From Replications to Revelations: Heteroskedasticity-Robust Inference," MPRA Paper 122724, University Library of Munich, Germany.
    10. Tom Boot & Gianmaria Niccodemi & Tom Wansbeek, 2023. "Unbiased estimation of the OLS covariance matrix when the errors are clustered," Empirical Economics, Springer, vol. 64(6), pages 2511-2533, June.
    11. Hwang, Jungbin, 2021. "Simple and trustworthy cluster-robust GMM inference," Journal of Econometrics, Elsevier, vol. 222(2), pages 993-1023.
    12. Hansen, Bruce E. & Lee, Seojeong, 2019. "Asymptotic theory for clustered samples," Journal of Econometrics, Elsevier, vol. 210(2), pages 268-290.
    13. James G. MacKinnon, 2019. "How cluster-robust inference is changing applied econometrics," Canadian Journal of Economics, Canadian Economics Association, vol. 52(3), pages 851-881, August.
    14. Bruno Ferman, 2023. "Inference in difference‐in‐differences: How much should we trust in independent clusters?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(3), pages 358-369, April.
    15. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    16. Pötscher, Benedikt M. & Preinerstorfer, David, 2021. "Valid Heteroskedasticity Robust Testing," MPRA Paper 117855, University Library of Munich, Germany, revised Jul 2023.
    17. MacKinnon, James G., 2023. "Using large samples in econometrics," Journal of Econometrics, Elsevier, vol. 235(2), pages 922-926.
    18. Djogbenou, Antoine A. & MacKinnon, James G. & Nielsen, Morten Ørregaard, 2019. "Asymptotic theory and wild bootstrap inference with clustered errors," Journal of Econometrics, Elsevier, vol. 212(2), pages 393-412.
    19. MacKinnon, James G. & Webb, Matthew D., 2020. "Randomization inference for difference-in-differences with few treated clusters," Journal of Econometrics, Elsevier, vol. 218(2), pages 435-450.
    20. James G. MacKinnon & Matthew D. Webb, 2017. "Pitfalls When Estimating Treatment Effects Using Clustered Data," Working Paper 1387, Economics Department, Queen's University.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:japmet:v:40:y:2025:i:3:p:291-309. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.interscience.wiley.com/jpages/0883-7252/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.