IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2506.07140.html
   My bibliography  Save this paper

Quantile-Optimal Policy Learning under Unmeasured Confounding

Author

Listed:
  • Zhongren Chen
  • Siyu Chen
  • Zhengling Qi
  • Xiaohong Chen
  • Zhuoran Yang

Abstract

We study quantile-optimal policy learning where the goal is to find a policy whose reward distribution has the largest $\alpha$-quantile for some $\alpha \in (0, 1)$. We focus on the offline setting whose generating process involves unobserved confounders. Such a problem suffers from three main challenges: (i) nonlinearity of the quantile objective as a functional of the reward distribution, (ii) unobserved confounding issue, and (iii) insufficient coverage of the offline dataset. To address these challenges, we propose a suite of causal-assisted policy learning methods that provably enjoy strong theoretical guarantees under mild conditions. In particular, to address (i) and (ii), using causal inference tools such as instrumental variables and negative controls, we propose to estimate the quantile objectives by solving nonlinear functional integral equations. Then we adopt a minimax estimation approach with nonparametric models to solve these integral equations, and propose to construct conservative policy estimates that address (iii). The final policy is the one that maximizes these pessimistic estimates. In addition, we propose a novel regularized policy learning method that is more amenable to computation. Finally, we prove that the policies learned by these methods are $\tilde{\mathscr{O}}(n^{-1/2})$ quantile-optimal under a mild coverage assumption on the offline dataset. Here, $\tilde{\mathscr{O}}(\cdot)$ omits poly-logarithmic factors. To the best of our knowledge, we propose the first sample-efficient policy learning algorithms for estimating the quantile-optimal policy when there exist unmeasured confounding.

Suggested Citation

  • Zhongren Chen & Siyu Chen & Zhengling Qi & Xiaohong Chen & Zhuoran Yang, 2025. "Quantile-Optimal Policy Learning under Unmeasured Confounding," Papers 2506.07140, arXiv.org.
  • Handle: RePEc:arx:papers:2506.07140
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2506.07140
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lee, Sokbae, 2003. "Efficient Semiparametric Estimation Of A Partially Linear Quantile Regression Model," Econometric Theory, Cambridge University Press, vol. 19(1), pages 1-31, February.
    2. Kevin Dowd & David Blake, 2006. "After VaR: The Theory, Estimation, and Insurance Applications of Quantile‐Based Risk Measures," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 73(2), pages 193-229, June.
    3. Xiaohong Chen & Demian Pouzo, 2015. "Sieve Wald and QLR Inferences on Semi/Nonparametric Conditional Moment Models," Econometrica, Econometric Society, vol. 83(3), pages 1013-1079, May.
    4. Xiaohong Chen & Victor Chernozhukov & Sokbae Lee & Whitney K. Newey, 2014. "Local Identification of Nonparametric and Semiparametric Models," Econometrica, Econometric Society, vol. 82(2), pages 785-809, March.
    5. Ai, Chunrong & Chen, Xiaohong, 2012. "The semiparametric efficiency bound for models of sequential moment restrictions containing unknown functions," Journal of Econometrics, Elsevier, vol. 170(2), pages 442-457.
    6. Xiaohong Chen & Demian Pouzo, 2012. "Estimation of Nonparametric Conditional Moment Models With Possibly Nonsmooth Generalized Residuals," Econometrica, Econometric Society, vol. 80(1), pages 277-321, January.
    7. Wu, Tracy Z. & Yu, Keming & Yu, Yan, 2010. "Single-index quantile regression," Journal of Multivariate Analysis, Elsevier, vol. 101(7), pages 1607-1621, August.
    8. Alberto Abadie & Joshua Angrist & Guido Imbens, 2002. "Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings," Econometrica, Econometric Society, vol. 70(1), pages 91-117, January.
    9. Asaf Cassel & Shie Mannor & Assaf Zeevi, 2023. "A General Framework for Bandit Problems Beyond Cumulative Objectives," Mathematics of Operations Research, INFORMS, vol. 48(4), pages 2196-2232, November.
    10. Yuanjia Wang & Haoda Fu & Donglin Zeng, 2018. "Learning Optimal Personalized Treatment Rules in Consideration of Benefit and Risk: With an Application to Treating Type 2 Diabetes Patients With Insulin Therapies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(521), pages 1-13, January.
    11. Xiaohong Chen & Oliver Linton & Ingrid Van Keilegom, 2003. "Estimation of Semiparametric Models when the Criterion Function Is Not Smooth," Econometrica, Econometric Society, vol. 71(5), pages 1591-1608, September.
    12. Gagliardini, Patrick & Scaillet, Olivier, 2012. "Tikhonov regularization for nonparametric instrumental variable estimators," Journal of Econometrics, Elsevier, vol. 167(1), pages 61-75.
    13. Rui Miao & Zhengling Qi & Cong Shi & Lin Lin, 2023. "Personalized Pricing with Invalid Instrumental Variables: Identification, Estimation, and Policy Learning," Papers 2302.12670, arXiv.org.
    14. Patrick Gagliardini & Olivier Scaillet, 2012. "Nonparametric Instrumental Variable Estimation of Structural Quantile Effects," Econometrica, Econometric Society, vol. 80(4), pages 1533-1562, July.
    15. Ethan X. Fang & Zhaoran Wang & Lan Wang, 2023. "Fairness-Oriented Learning for Optimal Individualized Treatment Rules," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(543), pages 1733-1746, July.
    16. Victor Chernozhukov & Christian Hansen, 2005. "An IV Model of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 73(1), pages 245-261, January.
    17. Joel L. Horowitz, 2007. "Asymptotic Normality Of A Nonparametric Instrumental Variables Estimator," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 48(4), pages 1329-1349, November.
    18. Chernozhukov, Victor & Hansen, Christian, 2008. "Instrumental variable quantile regression: A robust inference approach," Journal of Econometrics, Elsevier, vol. 142(1), pages 379-398, January.
    19. Whitney K. Newey & James L. Powell, 2003. "Instrumental Variable Estimation of Nonparametric Models," Econometrica, Econometric Society, vol. 71(5), pages 1565-1578, September.
    20. Chunrong Ai & Xiaohong Chen, 2003. "Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions," Econometrica, Econometric Society, vol. 71(6), pages 1795-1843, November.
    21. Joel L. Horowitz & Sokbae Lee, 2007. "Nonparametric Instrumental Variables Estimation of a Quantile Regression Model," Econometrica, Econometric Society, vol. 75(4), pages 1191-1208, July.
    22. Wang Miao & Zhi Geng & Eric J Tchetgen Tchetgen, 2018. "Identifying causal effects with proxy variables of an unmeasured confounder," Biometrika, Biometrika Trust, vol. 105(4), pages 987-993.
    23. Chamberlain, Gary, 1992. "Efficiency Bounds for Semiparametric Regression," Econometrica, Econometric Society, vol. 60(3), pages 567-596, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Xiaohong Chen & Demian Pouzo, 2015. "Sieve Wald and QLR Inferences on Semi/Nonparametric Conditional Moment Models," Econometrica, Econometric Society, vol. 83(3), pages 1013-1079, May.
    2. Chen, Xiaohong & Pouzo, Demian & Powell, James L., 2019. "Penalized sieve GEL for weighted average derivatives of nonparametric quantile IV regressions," Journal of Econometrics, Elsevier, vol. 213(1), pages 30-53.
    3. Xiaohong Chen & Demian Pouzo, 2014. "Sieve Wald and QLR Inferences on Semi/nonparametric Conditional Moment Models," CeMMAP working papers 38/14, Institute for Fiscal Studies.
    4. Victor Chernozhukov & Whitney K. Newey & Andres Santos, 2023. "Constrained Conditional Moment Restriction Models," Econometrica, Econometric Society, vol. 91(2), pages 709-736, March.
    5. Hiroaki Kaido & Kaspar Wüthrich, 2021. "Decentralization estimators for instrumental variable quantile regression models," Quantitative Economics, Econometric Society, vol. 12(2), pages 443-475, May.
    6. Christoph Breunig, 2019. "Specification Testing in Nonparametric Instrumental Quantile Regression," Papers 1909.10129, arXiv.org.
    7. Xiaohong Chen & Demian Pouzo, 2012. "Estimation of Nonparametric Conditional Moment Models With Possibly Nonsmooth Generalized Residuals," Econometrica, Econometric Society, vol. 80(1), pages 277-321, January.
    8. Kaspar Wüthrich, 2020. "A Comparison of Two Quantile Models With Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 443-456, April.
    9. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    10. Xiaohong Chen & Victor Chernozhukov & Sokbae Lee & Whitney K. Newey, 2014. "Local Identification of Nonparametric and Semiparametric Models," Econometrica, Econometric Society, vol. 82(2), pages 785-809, March.
    11. Victor Chernozhukov & Christian Hansen & Kaspar Wuthrich, 2020. "Instrumental Variable Quantile Regression," Papers 2009.00436, arXiv.org.
    12. Jean-Jacques Forneron, 2019. "A Sieve-SMM Estimator for Dynamic Models," Papers 1902.01456, arXiv.org, revised Jan 2023.
    13. V. Chernozhukov & C. Hansen, 2013. "Quantile Models with Endogeneity," Annual Review of Economics, Annual Reviews, vol. 5(1), pages 57-81, May.
    14. repec:hum:wpaper:sfb649dp2016-032 is not listed on IDEAS
    15. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    16. Wüthrich, Kaspar, 2019. "A closed-form estimator for quantile treatment effects with endogeneity," Journal of Econometrics, Elsevier, vol. 210(2), pages 219-235.
    17. Marina Dias & Demian Pouzo, 2021. "Inference for multi-valued heterogeneous treatment effects when the number of treated units is small," Papers 2105.10965, arXiv.org.
    18. Breunig, Christoph, 2016. "Specification testing in nonparametric instrumental quantile regression," SFB 649 Discussion Papers 2016-032, Humboldt University Berlin, Collaborative Research Center 649: Economic Risk.
    19. Chen, Xiaohong & Pouzo, Demian, 2009. "Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals," Journal of Econometrics, Elsevier, vol. 152(1), pages 46-60, September.
    20. Hidehiko Ichimura & Whitney K. Newey, 2022. "The influence function of semiparametric estimators," Quantitative Economics, Econometric Society, vol. 13(1), pages 29-61, January.
    21. Pereda-Fernández, Santiago, 2023. "Identification and estimation of triangular models with a binary treatment," Journal of Econometrics, Elsevier, vol. 234(2), pages 585-623.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2506.07140. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.