IDEAS home Printed from https://ideas.repec.org/p/cpb/discus/452.html
   My bibliography  Save this paper

Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences

Author

Listed:
  • Mark Kattenberg

    (CPB Netherlands Bureau for Economic Policy Analysis)

  • Bas Scheer

    (CPB Netherlands Bureau for Economic Policy Analysis)

  • Jurre Thiel

    (CPB Netherlands Bureau for Economic Policy Analysis)

Abstract

Recently developed heterogeneity-robust two-way fixed effects (TWFE) estimators do not quantify the full heterogeneity in treatment effects in a difference-in-differences research design. We therefore present a computationally feasible algorithm to estimate heterogeneous treatment effects in the presence of many fixed effects using causal forests. Our modification identifies treatment effects by partialling out fixed effect using group averages. Simulation results suggest that our algorithm provides consistent estimates of the Conditional Average Treatment effect for the Treated in a (staggered) difference-in-differences research design. Finally, we use our method to document heterogeneity in the treatment effect of alternative work arrangements (payrolling) on hourly wages. We find evidence that wages fell by 3.7 percent in the first year of payrolling for a specific subgroup of workers only. Both conclusions did not appear in a conventional heterogeneity analysis using manual subgroups. The R-code of our algorithm is publicly available online.

Suggested Citation

  • Mark Kattenberg & Bas Scheer & Jurre Thiel, 2023. "Causal forests with fixed effects for treatment effect heterogeneity in difference-in-differences," CPB Discussion Paper 452, CPB Netherlands Bureau for Economic Policy Analysis.
    • Handle: RePEc:cpb:discus:452
    DOI: 10.34932/216c-yz58
    as

    Download full text from publisher

    File URL: https://www.cpb.nl/sites/default/files/omnidownload/CPB-Discussion-Paper-452-Causal-forests-with-fixed-effects-for-treatment-effect-heterogeneity-in-difference-in-differences.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.34932/216c-yz58?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Andres Drenik & Simon Jager & Pascuel Plotkin & Benjamin Schoefer, 2023. "Paying Outsourced Labor: Direct Evidence from Linked Temp Agency-Worker-Client Data," The Review of Economics and Statistics, MIT Press, vol. 105(1), pages 206-216, January.
    2. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    3. Clément de Chaisemartin & Xavier D'Haultfœuille, 2020. "Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects," American Economic Review, American Economic Association, vol. 110(9), pages 2964-2996, September.
    4. de Chaisemartin, Clément & D’Haultfœuille, Xavier, 2023. "Two-way fixed effects and differences-in-differences estimators with several treatments," Journal of Econometrics, Elsevier, vol. 236(2).
    5. J. Michelle Brock & Ralph De Haas, 2023. "Discriminatory Lending: Evidence from Bankers in the Lab," American Economic Journal: Applied Economics, American Economic Association, vol. 15(2), pages 31-68, April.
    6. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    7. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    8. Deborah Goldschmidt & Johannes F. Schmieder, 2017. "The Rise of Domestic Outsourcing and the Evolution of the German Wage Structure," The Quarterly Journal of Economics, Oxford University Press, vol. 132(3), pages 1165-1217.
    9. Gavrilova, Evelina & Langørgen, Audun & Zoutman, Floris T., 2023. "Dynamic Causal Forests, with an Application to Payroll Tax Incidence in Norway," Discussion Papers 2023/9, Norwegian School of Economics, Department of Business and Management Science.
    10. John M. Abowd & Francis Kramarz & David N. Margolis, 1999. "High Wage Workers and High Wage Firms," Econometrica, Econometric Society, vol. 67(2), pages 251-334, March.
    11. Michael C Knaus, 2022. "Double machine learning-based programme evaluation under unconfoundedness [Econometric methods for program evaluation]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 602-627.
    12. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    13. Julia Hatamyar & Noemi Kreif & Rudi Rocha & Martin Huber, 2023. "Machine Learning for Staggered Difference-in-Differences and Dynamic Treatment Effect Heterogeneity," Papers 2310.11962, arXiv.org.
    14. Stephen G. Donald & Kevin Lang, 2007. "Inference with Difference-in-Differences and Other Panel Data," The Review of Economics and Statistics, MIT Press, vol. 89(2), pages 221-233, May.
    15. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    16. Jonathan M.V. Davis & Sara B. Heller, 2020. "Rethinking the Benefits of Youth Employment Programs: The Heterogeneous Effects of Summer Jobs," The Review of Economics and Statistics, MIT Press, vol. 102(4), pages 664-677, October.
    17. Abraham Katharine G. & Amaya Ashley, 2019. "Probing for Informal Work Activity," Journal of Official Statistics, Sciendo, vol. 35(3), pages 487-508, September.
    18. Lu Tian & Ash A. Alizadeh & Andrew J. Gentles & Robert Tibshirani, 2014. "A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1517-1532, December.
    19. Jordà , Òscar & Dube, Arindrajit & Girardi, Daniele & Taylor, Alan, 2023. "A Local Projections Approach to Difference-in-Differences Event Studies," CEPR Discussion Papers 18141, C.E.P.R. Discussion Papers.
    20. Somaini Paulo & Wolak Frank A., 2016. "An Algorithm to Estimate the Two-Way Fixed Effects Model," Journal of Econometric Methods, De Gruyter, vol. 5(1), pages 143-152, January.
    21. Alberto Abadie, 2005. "Semiparametric Difference-in-Differences Estimators," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 72(1), pages 1-19.
    22. Neng-Chieh Chang, 2020. "Double/debiased machine learning for difference-in-differences models," The Econometrics Journal, Royal Economic Society, vol. 23(2), pages 177-191.
    23. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey, 2017. "Double/Debiased/Neyman Machine Learning of Treatment Effects," American Economic Review, American Economic Association, vol. 107(5), pages 261-265, May.
    24. Susan Athey & Guido W. Imbens, 2006. "Identification and Inference in Nonlinear Difference-in-Differences Models," Econometrica, Econometric Society, vol. 74(2), pages 431-497, March.
    25. Athey, Susan & Imbens, Guido W., 2019. "Machine Learning Methods Economists Should Know About," Research Papers 3776, Stanford University, Graduate School of Business.
    26. Callaway, Brantly & Sant’Anna, Pedro H.C., 2021. "Difference-in-Differences with multiple time periods," Journal of Econometrics, Elsevier, vol. 225(2), pages 200-230.
    27. Evelina Gavrilova & Audun Langørgen & Floris T. Zoutman & Floris Zoutman, 2023. "Dynamic Causal Forests, with an Application to Payroll Tax Incidence in Norway," CESifo Working Paper Series 10532, CESifo.
    28. Tito Boeri & Giulia Giupponi & Alan B. Krueger & Stephen Machin, 2020. "Solo Self-Employment and Alternative Work Arrangements: A Cross-Country Perspective on the Changing Composition of Jobs," Journal of Economic Perspectives, American Economic Association, vol. 34(1), pages 170-195, Winter.
    29. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    30. Kayo Murakami & Hideki Shimada & Yoshiaki Ushifusa & Takanori Ida, 2022. "Heterogeneous Treatment Effects Of Nudge And Rebate: Causal Machine Learning In A Field Experiment On Electricity Conservation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 63(4), pages 1779-1803, November.
    31. Xueming Luo & Xianghua Lu & Jing Li, 2019. "When and How to Leverage E-commerce Cart Targeting: The Relative and Moderated Effects of Scarcity and Price Incentives with a Two-Stage Field Experiment and Causal Forest Optimization," Information Systems Research, INFORMS, vol. 30(4), pages 1203-1227, December.
    32. Hoffman, Ian & Mast, Evan, 2019. "Heterogeneity in the effect of federal spending on local crime: Evidence from causal forests," Regional Science and Urban Economics, Elsevier, vol. 78(C).
    33. Victor Chernozhukov & Mert Demirer & Esther Duflo & Iván Fernández-Val, 2018. "Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, with an Application to Immunization in India," NBER Working Papers 24678, National Bureau of Economic Research, Inc.
    34. Lawrence F. Katz & Alan B. Krueger, 2019. "The Rise and Nature of Alternative Work Arrangements in the United States, 1995–2015," ILR Review, Cornell University, ILR School, vol. 72(2), pages 382-416, March.
    35. Miller, Steve, 2020. "Causal forest estimation of heterogeneous and time-varying environmental policy effects," Journal of Environmental Economics and Management, Elsevier, vol. 103(C).
    36. Susan Athey & Guido W. Imbens, 2019. "Machine Learning Methods That Economists Should Know About," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 685-725, August.
    37. Donald B. Rubin, 2005. "Causal Inference Using Potential Outcomes: Design, Modeling, Decisions," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 322-331, March.
    38. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    39. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2016. "Double/Debiased Machine Learning for Treatment and Causal Parameters," Papers 1608.00060, arXiv.org, revised Dec 2017.
    40. Jonathan Roth, 2022. "Pretest with Caution: Event-Study Estimates after Testing for Parallel Trends," American Economic Review: Insights, American Economic Association, vol. 4(3), pages 305-322, September.
    41. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    42. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Mar 2024.
    2. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    3. Gregory Faletto, 2023. "Fused Extended Two-Way Fixed Effects for Difference-in-Differences with Staggered Adoptions," Papers 2312.05985, arXiv.org.
    4. Athey, Susan & Imbens, Guido W., 2022. "Design-based analysis in Difference-In-Differences settings with staggered adoption," Journal of Econometrics, Elsevier, vol. 226(1), pages 62-79.
    5. Bas Scheer & Wiljan van den Berge & Maarten Goos & Alan Manning & Anna Salomons, 2022. "Alternative Work Arrangements and Worker Outcomes: Evidence from Payrolling," CPB Discussion Paper 435, CPB Netherlands Bureau for Economic Policy Analysis.
    6. Michael Lechner, 2023. "Causal Machine Learning and its use for public policy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 159(1), pages 1-15, December.
    7. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    8. Anna Baiardi & Andrea A. Naghi, 2021. "The Value Added of Machine Learning to Causal Inference: Evidence from Revisited Studies," Papers 2101.00878, arXiv.org.
    9. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    10. Anna Baiardi & Andrea A. Naghi, 2021. "The Value Added of Machine Learning to Causal Inference: Evidence from Revisited Studies," Tinbergen Institute Discussion Papers 21-001/V, Tinbergen Institute.
    11. Daniel Goller, 2023. "Analysing a built-in advantage in asymmetric darts contests using causal machine learning," Annals of Operations Research, Springer, vol. 325(1), pages 649-679, June.
    12. Miquel Oliu-Barton & Bary S. R. Pradelski & Nicolas Woloszko & Lionel Guetta-Jeanrenaud & Philippe Aghion & Patrick Artus & Arnaud Fontanet & Philippe Martin & Guntram B. Wolff, 2022. "The effect of COVID certificates on vaccine uptake, health outcomes, and the economy," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    13. Michael C Knaus, 2022. "Double machine learning-based programme evaluation under unconfoundedness [Econometric methods for program evaluation]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 602-627.
    14. Diego Daruich & Sabrina Di Addario & Raffaele Saggio, 2023. "The Effects of Partial Employment Protection Reforms: Evidence from Italy," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(6), pages 2880-2942.
    15. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    16. Phillip Heiler & Michael C. Knaus, 2021. "Effect or Treatment Heterogeneity? Policy Evaluation with Aggregated and Disaggregated Treatments," Papers 2110.01427, arXiv.org, revised Aug 2023.
    17. Valente, Marica, 2023. "Policy evaluation of waste pricing programs using heterogeneous causal effect estimation," Journal of Environmental Economics and Management, Elsevier, vol. 117(C).
    18. Falco J. Bargagli Stoffi & Kenneth De Beckker & Joana E. Maldonado & Kristof De Witte, 2021. "Assessing Sensitivity of Machine Learning Predictions.A Novel Toolbox with an Application to Financial Literacy," Papers 2102.04382, arXiv.org.
    19. Christian Aleman & Christopher Busch & Alexander Ludwig & Raul Santaeulalia-Llopis, 2022. "A Stage-Based Identification of Policy Effects," PIER Working Paper Archive 22-026, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    20. Goller, Daniel & Harrer, Tamara & Lechner, Michael & Wolff, Joachim, 2021. "Active labour market policies for the long-term unemployed: New evidence from causal machine learning," Economics Working Paper Series 2108, University of St. Gallen, School of Economics and Political Science.

    More about this item

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C23 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Models with Panel Data; Spatio-temporal Models
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cpb:discus:452. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/cpbgvnl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.