IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2506.12765.html

Model Risk in Machine-Learning Distributional IV Estimation

Author

Listed:
  • Charles Shaw

Abstract

We study model risk in machine-learning estimation of the Distributional Instrumental Variable Local Average Treatment Effect (D-IV-LATE), the distributional IV effect for the subpopulation induced into treatment by the instrument. The contribution is not a new neural causal estimand. We implement a reduced-form orthogonal level-score DML estimator for the covariate-adjusted D-IV-LATE target and use it to ask how much the nuisance learner matters for distributional IV conclusions. In simulations with explicit monotone principal strata and known complier truth, Kolmogorov-Arnold Networks (KANs) are faster than Random Forests in every scenario examined, but Random Forests usually produce more accurate D-IV-LATE curves. A targeted KAN ablation selects a width-64 KAN as the best KAN variant among those tested, but this is a speed/accuracy tradeoff rather than evidence of KAN dominance. In a 401(k) application, RF and KAN estimates differ materially, with frequent sign reversals along the estimated curve. The KAN instrument-propensity estimates also concentrate near the boundaries, so the KAN empirical curve is best read as sensitivity evidence. In inference validation, KAN pointwise intervals undercover badly under both asymptotic and bootstrap constructions, while RF asymptotic intervals are better calibrated in the validation designs. The main lesson is cautionary and constructive: speed and architectural flexibility are not enough for causal inference. Applied researchers using ML-based distributional IV estimators would be advised to benchmark nuisance learners, report overlap and calibration diagnostics, and validate inference directly.

Suggested Citation

  • Charles Shaw, 2025. "Model Risk in Machine-Learning Distributional IV Estimation," Papers 2506.12765, arXiv.org, revised Jun 2026.
  • Handle: RePEc:arx:papers:2506.12765
    as

    Download full text from publisher

    File URL: https://arxiv.org/pdf/2506.12765
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    2. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    3. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    4. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "Inference on Treatment Effects after Selection among High-Dimensional Controlsâ€," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(2), pages 608-650.
    5. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    6. Guido W. Imbens & Donald B. Rubin, 1997. "Estimating Outcome Distributions for Compliers in Instrumental Variables Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 555-574.
    7. Abadie A., 2002. "Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable Models," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 284-292, March.
    8. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    9. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Shota Yasui & Tatsushi Oka & Undral Byambadalai & Yuki Oishi, 2026. "Distributional Treatment Effects of Content Promotion: Evidence from an ABEMA Field Experiment," Papers 2601.11185, arXiv.org.
    2. Mogstad, Magne & Torgovitsky, Alexander, 2024. "Instrumental variables with unobserved heterogeneity in treatment effects," Handbook of Labor Economics,, Elsevier.
    3. Vira Semenova, 2020. "Generalized Lee Bounds," Papers 2008.12720, arXiv.org, revised May 2025.
    4. Abadie, Alberto, 2003. "Semiparametric instrumental variable estimation of treatment response models," Journal of Econometrics, Elsevier, vol. 113(2), pages 231-263, April.
    5. Sergio Firpo & Cristine Pinto, 2016. "Identification and Estimation of Distributional Impacts of Interventions Using Changes in Inequality Measures," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(3), pages 457-486, April.
    6. Haitian Xie, 2020. "Efficient and Robust Estimation of the Generalized LATE Model," Papers 2001.06746, arXiv.org, revised Feb 2022.
    7. Undral Byambadalai & Tomu Hirata & Tatsushi Oka & Shota Yasui, 2025. "Beyond the Average: Distributional Causal Inference under Imperfect Compliance," Papers 2509.15594, arXiv.org, revised Oct 2025.
    8. Joshua D. Angrist, 2022. "Empirical Strategies in Economics: Illuminating the Path From Cause to Effect," Econometrica, Econometric Society, vol. 90(6), pages 2509-2539, November.
    9. Su, Liangjun & Ura, Takuya & Zhang, Yichong, 2019. "Non-separable models with high-dimensional data," Journal of Econometrics, Elsevier, vol. 212(2), pages 646-677.
    10. Alberto Abadie, 2000. "Semiparametric Estimation of Instrumental Variable Models for Causal Effects," NBER Technical Working Papers 0260, National Bureau of Economic Research, Inc.
    11. Firpo, Sergio Pinheiro & Pinto, Rafael de Carvalho Cayres, 2012. "Combining Strategies for the Estimation of Treatment Effects," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 32(1), March.
    12. Huber Martin & Wüthrich Kaspar, 2019. "Local Average and Quantile Treatment Effects Under Endogeneity: A Review," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-27, January.
    13. Simon Calmar Andersen & Louise Beuchert & Phillip Heiler & Helena Skyt Nielsen, 2023. "A Guide to Impact Evaluation under Sample Selection and Missing Data: Teacher's Aides and Adolescent Mental Health," Papers 2308.04963, arXiv.org.
    14. Esfandiar Maasoumi & Daniel L. Millimet & Dipanwita Sarkar, 2009. "Who Benefits from Marriage?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 71(1), pages 1-33, February.
    15. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    16. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    17. Pedro H. C. Sant'Anna, 2016. "Program Evaluation with Right-Censored Data," Papers 1604.02642, arXiv.org.
    18. Kaspar Wüthrich, 2020. "A Comparison of Two Quantile Models With Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 443-456, April.
    19. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    20. Salima BOUAYAD AGHA & Nadine TURPIN & Lionel VEDRINE, 2013. "Au-Delà De La Moyenne : Les Effets Par Quantile De La Politique De Cohésion De L’Union Européenne," Region et Developpement, Region et Developpement, LEAD, Universite du Sud - Toulon Var, vol. 38, pages 27-48.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2506.12765. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: https://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.