IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2410.07443.html
   My bibliography  Save this paper

On the Lower Confidence Band for the Optimal Welfare in Policy Learning

Author

Listed:
  • Kirill Ponomarev
  • Vira Semenova

Abstract

We study inference on the optimal welfare in a policy learning problem and propose reporting a lower confidence band (LCB). A natural approach to constructing an LCB is to invert a one-sided t-test based on an efficient estimator for the optimal welfare. However, we show that for an empirically relevant class of DGPs, such an LCB can be first-order dominated by an LCB based on a welfare estimate for a suitable suboptimal treatment policy. We show that such first-order dominance is possible if and only if the optimal treatment policy is not ``well-separated'' from the rest, in the sense of the commonly imposed margin condition. When this condition fails, standard debiased inference methods are not applicable. We show that uniformly valid and easy-to-compute LCBs can be constructed analytically by inverting moment-inequality tests with the maximum and quasi-likelihood-ratio test statistics. As an empirical illustration, we revisit the National JTPA study and find that the proposed LCBs achieve reliable coverage and competitive length.

Suggested Citation

  • Kirill Ponomarev & Vira Semenova, 2024. "On the Lower Confidence Band for the Optimal Welfare in Policy Learning," Papers 2410.07443, arXiv.org, revised Sep 2025.
    • Handle: RePEc:arx:papers:2410.07443
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2410.07443
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Keisuke Hirano & Jack R. Porter, 2009. "Asymptotics for Statistical Treatment Rules," Econometrica, Econometric Society, vol. 77(5), pages 1683-1701, September.
    2. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    3. Alberto Abadie & Joshua Angrist & Guido Imbens, 2002. "Instrumental Variables Estimates of the Effect of Subsidized Training on the Quantiles of Trainee Earnings," Econometrica, Econometric Society, vol. 70(1), pages 91-117, January.
    4. Victor Chernozhukov & Denis Chetverikov & Kengo Kato & Aureo de Paula, 2019. "Inference on Causal and Structural Parameters using Many Moment Inequalities," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(5), pages 1867-1900.
    5. Joseph P. Romano & Azeem M. Shaikh & Michael Wolf, 2014. "A Practical Two‐Step Method for Testing Moment Inequalities," Econometrica, Econometric Society, vol. 82, pages 1979-2002, September.
    6. Tetenov, Aleksey, 2012. "Statistical treatment choice based on asymmetric minimax regret criteria," Journal of Econometrics, Elsevier, vol. 166(1), pages 157-165.
    7. Matthew S. Johnson & David I. Levine & Michael W. Toffel, 2023. "Improving Regulatory Effectiveness through Better Targeting: Evidence from OSHA," American Economic Journal: Applied Economics, American Economic Association, vol. 15(4), pages 30-67, October.
    8. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    9. Charles F. Manski, 2004. "Statistical Treatment Rules for Heterogeneous Populations," Econometrica, Econometric Society, vol. 72(4), pages 1221-1246, July.
    10. Liyang Sun, 2021. "Empirical Welfare Maximization with Constraints," Papers 2103.15298, arXiv.org, revised Sep 2024.
    11. Keisuke Hirano & Jack R. Porter, 2012. "Impossibility Results for Nondifferentiable Functionals," Econometrica, Econometric Society, vol. 80(4), pages 1769-1790, July.
    12. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    13. Eric Mbakop & Max Tabord‐Meehan, 2021. "Model Selection for Treatment Choice: Penalized Welfare Maximization," Econometrica, Econometric Society, vol. 89(2), pages 825-848, March.
    14. Donald W. K. Andrews & Gustavo Soares, 2010. "Inference for Parameters Defined by Moment Inequalities Using Generalized Moment Selection," Econometrica, Econometric Society, vol. 78(1), pages 119-157, January.
    15. Stoye, Jörg, 2009. "Minimax regret treatment choice with finite samples," Journal of Econometrics, Elsevier, vol. 151(1), pages 70-81, July.
    16. Dehejia, Rajeev H., 2005. "Program evaluation as a decision problem," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 141-173.
    17. Bhattacharya, Debopam & Dupas, Pascaline, 2012. "Inferring welfare maximizing treatment assignment under budget constraints," Journal of Econometrics, Elsevier, vol. 167(1), pages 168-196.
    18. Howard S. Bloom & Larry L. Orr & Stephen H. Bell & George Cave & Fred Doolittle & Winston Lin & Johannes M. Bos, 1997. "The Benefits and Costs of JTPA Title II-A Programs: Key Findings from the National Job Training Partnership Act Study," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 549-576.
    19. Levy Jonathan & van der Laan Mark & Hubbard Alan & Pirracchio Romain, 2021. "A fundamental measure of treatment effect heterogeneity," Journal of Causal Inference, De Gruyter, vol. 9(1), pages 83-108, January.
    20. Athey, Susan & Keleher, Niall & Spiess, Jann, 2025. "Machine learning who to nudge: Causal vs predictive targeting in a field experiment on student financial aid renewal," Journal of Econometrics, Elsevier, vol. 249(PC).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Toru Kitagawa & Hugo Lopez & Jeff Rowley, 2022. "Stochastic Treatment Choice with Empirical Welfare Updating," Papers 2211.01537, arXiv.org, revised Feb 2023.
    2. Eric Mbakop & Max Tabord‐Meehan, 2021. "Model Selection for Treatment Choice: Penalized Welfare Maximization," Econometrica, Econometric Society, vol. 89(2), pages 825-848, March.
    3. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    4. Shosei Sakaguchi, 2021. "Estimation of Optimal Dynamic Treatment Assignment Rules under Policy Constraints," Papers 2106.05031, arXiv.org, revised Aug 2024.
    5. Yuya Sasaki & Takuya Ura, 2020. "Welfare Analysis via Marginal Treatment Effects," Papers 2012.07624, arXiv.org.
    6. Timothy B. Armstrong & Shu Shen, 2023. "Inference on optimal treatment assignments," The Japanese Economic Review, Springer, vol. 74(4), pages 471-500, October.
    7. Toru Kitagawa & Aleksey Tetenov, 2018. "Who Should Be Treated? Empirical Welfare Maximization Methods for Treatment Choice," Econometrica, Econometric Society, vol. 86(2), pages 591-616, March.
    8. Toru Kitagawa & Aleksey Tetenov, 2017. "Equality-minded treatment choice," CeMMAP working papers 10/17, Institute for Fiscal Studies.
    9. Toru Kitagawa & Guanyi Wang, 2021. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," CeMMAP working papers CWP28/21, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    10. Toru Kitagawa & Aleksey Tetenov, 2021. "Equality-Minded Treatment Choice," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(2), pages 561-574, March.
    11. Kitagawa, Toru & Wang, Guanyi, 2023. "Who should get vaccinated? Individualized allocation of vaccines over SIR network," Journal of Econometrics, Elsevier, vol. 232(1), pages 109-131.
    12. Hirano, Keisuke & Porter, Jack R., 2020. "Asymptotic analysis of statistical decision rules in econometrics," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 283-354, Elsevier.
    13. Toru Kitagawa & Aleksey Tetenov, 2015. "Who should be treated? Empirical welfare maximization methods for treatment choice," CeMMAP working papers 10/15, Institute for Fiscal Studies.
    14. Toru Kitagawa & Sokbae Lee & Chen Qiu, 2022. "Treatment Choice with Nonlinear Regret," Papers 2205.08586, arXiv.org, revised Oct 2024.
    15. Garbero, Alessandra & Sakos, Grayson & Cerulli, Giovanni, 2023. "Towards data-driven project design: Providing optimal treatment rules for development projects," Socio-Economic Planning Sciences, Elsevier, vol. 89(C).
    16. Undral Byambadalai, 2022. "Identification and Inference for Welfare Gains without Unconfoundedness," Papers 2207.04314, arXiv.org.
    17. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    18. Shosei Sakaguchi, 2025. "Estimation of optimal dynamic treatment assignment rules under policy constraints," Quantitative Economics, Econometric Society, vol. 16(3), pages 981-1022, July.
    19. Toru Kitagawa & Weining Wang & Mengshan Xu, 2022. "Policy Choice in Time Series by Empirical Welfare Maximization," Papers 2205.03970, arXiv.org, revised Nov 2025.
    20. Yu-Chang Chen & Haitian Xie, 2022. "Personalized Subsidy Rules," Papers 2202.13545, arXiv.org, revised Mar 2022.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2410.07443. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.