IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2404.01566.html

Heterogeneous Treatment Effects and Causal Mechanisms

Author

Listed:
  • Jiawei Fu
  • Tara Slough

Abstract

The credibility revolution advances the use of research designs that permit identification and estimation of causal effects. However, understanding which mechanisms produce measured causal effects remains a challenge. The dominant current approach to the quantitative evaluation of mechanisms relies on the detection of heterogeneous treatment effects (HTEs) with respect to pre-treatment covariates. This paper develops a framework to understand when the existence of such heterogeneous treatment effects can support inferences about the activation of a mechanism. We show first that this design cannot provide evidence of mechanism activation without additional, generally implicit, exclusion assumptions. Further, even when these assumptions are satisfied, the presence of HTEs supports the inference that mechanism is active but the absence of HTEs is generally uninformative about mechanism activation. We provide novel guidance for interpretation and research design in light of these findings.

Suggested Citation

  • Jiawei Fu & Tara Slough, 2024. "Heterogeneous Treatment Effects and Causal Mechanisms," Papers 2404.01566, arXiv.org, revised Nov 2025.
  • Handle: RePEc:arx:papers:2404.01566
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2404.01566
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Grimmer, Justin & Messing, Solomon & Westwood, Sean J., 2017. "Estimating Heterogeneous Treatment Effects and the Effects of Heterogeneous Treatments with Ensemble Methods," Political Analysis, Cambridge University Press, vol. 25(4), pages 413-434, October.
    2. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    3. Imai, Kosuke & Keele, Luke & Tingley, Dustin & Yamamoto, Teppei, 2011. "Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies," American Political Science Review, Cambridge University Press, vol. 105(4), pages 765-789, November.
    4. Imai, Kosuke & Yamamoto, Teppei, 2013. "Identification and Sensitivity Analysis for Multiple Causal Mechanisms: Revisiting Evidence from Framing Experiments," Political Analysis, Cambridge University Press, vol. 21(2), pages 141-171, April.
    5. Strezhnev, Anton & Kelley, Judith G. & Simmons, Beth A., 2021. "Testing for Negative Spillovers: Is Promoting Human Rights Really Part of the “Problem”?," International Organization, Cambridge University Press, vol. 75(1), pages 71-102, January.
    6. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    7. John A. List & Azeem M. Shaikh & Atom Vayalinkal, 2023. "Multiple testing with covariate adjustment in experimental economics," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(6), pages 920-939, September.
    8. Bueno De Mesquita, Ethan & Tyson, Scott A., 2020. "The Commensurability Problem: Conceptual Difficulties in Estimating the Effect of Behavior on Behavior," American Political Science Review, Cambridge University Press, vol. 114(2), pages 375-391, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ejeta, Tadesse Tolera & Bai, Xiuguang, 2026. "Does the adoption of agricultural green production technologies improve eco-efficiency? Evidence from wheat farmers in Ethiopia," Agricultural Systems, Elsevier, vol. 231(C).
    2. Parker Hevron, 2018. "Judicialization and Its Effects: Experiments as a Way Forward," Laws, MDPI, vol. 7(2), pages 1-21, May.
    3. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    4. Adel Daoud, 2020. "The wealth of nations and the health of populations: A quasi-experimental design of the impact of sovereign debt crises on child mortality," Papers 2012.14941, arXiv.org.
    5. Acharya, Avidit & Blackwell, Matthew & Sen, Maya, 2016. "Explaining Causal Findings Without Bias: Detecting and Assessing Direct Effects," American Political Science Review, Cambridge University Press, vol. 110(3), pages 512-529, August.
    6. Krisztián Pósch, 2021. "Prying Open the Black Box of Causality: A Causal Mediation Analysis Test of Procedural Justice Policing," Journal of Quantitative Criminology, Springer, vol. 37(1), pages 217-245, March.
    7. Tingley, Dustin & Yamamoto, Teppei & Hirose, Kentaro & Keele, Luke & Imai, Kosuke, 2014. "mediation: R Package for Causal Mediation Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 59(i05).
    8. Nicholas Weller & Jeb Barnes, 2016. "Pathway Analysis and the Search for Causal Mechanisms," Sociological Methods & Research, , vol. 45(3), pages 424-457, August.
    9. Ludvig Norman, 2025. "Reconsidering mechanisms in social science research: the varied uses of an explanatory metaphor," Quality & Quantity: International Journal of Methodology, Springer, vol. 59(2), pages 1695-1712, April.
    10. Adam C. Sales, 2017. "Review," Journal of Educational and Behavioral Statistics, , vol. 42(1), pages 69-84, February.
    11. repec:osf:metaar:9a3rw_v1 is not listed on IDEAS
    12. Cammett, Melani & Şaşmaz, Aytuğ, 2017. "Political Context, Organizational Mission, and the Quality of Social Services: Insights from the Health Sector in Lebanon," World Development, Elsevier, vol. 98(C), pages 120-132.
    13. Brown, Martin & Henchoz, Caroline & Spycher, Thomas, 2018. "Culture and financial literacy: Evidence from a within-country language border," Journal of Economic Behavior & Organization, Elsevier, vol. 150(C), pages 62-85.
    14. Jiawei Fu, 2024. "Extracting Mechanisms from Heterogeneous Effects: An Identification Strategy for Mediation Analysis," Papers 2403.04131, arXiv.org, revised May 2026.
    15. Gregory A. PORUMBESCU & Nicola BELLE & Maria CUCCINIELLO & Greta NASI, 2019. "Does Transparency Lead To Coproduction?," TAD 14 The disciplines and the study of Public Administration: Transatlantic perspectives in the margin of the 14th Administration and Public Management International Conference, Bucharest, June 6-18 6, Research Centre in Public Administration and Public Services, Bucharest, Romania.
    16. Ward, Jeffrey T. & Hartley, Richard D. & Tillyer, Rob, 2016. "Unpacking gender and racial/ethnic biases in the federal sentencing of drug offenders: A causal mediation approach," Journal of Criminal Justice, Elsevier, vol. 46(C), pages 196-206.
    17. Francesco Vidoli, 2026. "The Geography of Impact: Endogenous Spatial Clustering for Difference-in-Differences Estimation," Working Papers 2601, University of Urbino Carlo Bo, Department of Economics, Society & Politics - Scientific Committee - L. Stefanini & G. Travaglini, revised 2026.
    18. Anastasia Hernández-Alemán & Noelia Cruz-Pérez & Juan C. Santamarta, 2024. "The Average Direct, Indirect and Total Effects of Environmental Concern on Pro-Environmental Behavior," Land, MDPI, vol. 13(8), pages 1-26, August.
    19. Gay, Victor & Boehnke, Jörn, 2017. "The Missing Men: World War I and Female Labor Participation," MPRA Paper 77560, University Library of Munich, Germany.
    20. S. Erdem Aytaç & Luis Schiumerini & Susan Stokes, 2018. "Why Do People Join Backlash Protests? Lessons from Turkey," Journal of Conflict Resolution, Peace Science Society (International), vol. 62(6), pages 1205-1228, July.
    21. Walter Sosa-Escudero & Maria Victoria Anauati & Wendy Brau, 2022. "Poverty, Inequality and Development Studies with Machine Learning," Advanced Studies in Theoretical and Applied Econometrics, in: Felix Chan & László Mátyás (ed.), Econometrics with Machine Learning, chapter 0, pages 291-335, Springer.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2404.01566. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.