IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2306.11979.html
   My bibliography  Save this paper

Qini Curves for Multi-Armed Treatment Rules

Author

Listed:
  • Erik Sverdrup
  • Han Wu
  • Susan Athey
  • Stefan Wager

Abstract

Qini curves have emerged as an attractive and popular approach for evaluating the benefit of data-driven targeting rules for treatment allocation. We propose a generalization of the Qini curve to multiple costly treatment arms, that quantifies the value of optimally selecting among both units and treatment arms at different budget levels. We develop an efficient algorithm for computing these curves and propose bootstrap-based confidence intervals that are exact in large samples for any point on the curve. These confidence intervals can be used to conduct hypothesis tests comparing the value of treatment targeting using an optimal combination of arms with using just a subset of arms, or with a non-targeting assignment rule ignoring covariates, at different budget levels. We demonstrate the statistical performance in a simulation experiment and an application to treatment targeting for election turnout.

Suggested Citation

  • Erik Sverdrup & Han Wu & Susan Athey & Stefan Wager, 2023. "Qini Curves for Multi-Armed Treatment Rules," Papers 2306.11979, arXiv.org, revised Oct 2024.
    • Handle: RePEc:arx:papers:2306.11979
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2306.11979
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    2. Zhengyuan Zhou & Susan Athey & Stefan Wager, 2023. "Offline Multi-Action Policy Learning: Generalization and Optimization," Operations Research, INFORMS, vol. 71(1), pages 148-183, January.
    3. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    4. Kosuke Imai & Michael Lingzhi Li, 2023. "Experimental Evaluation of Individualized Treatment Rules," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 118(541), pages 242-256, January.
    5. George B. Dantzig, 1957. "Discrete-Variable Extremum Problems," Operations Research, INFORMS, vol. 5(2), pages 266-288, April.
    6. Hans Kellerer & Ulrich Pferschy & David Pisinger, 2004. "Knapsack Problems," Springer Books, Springer, number 978-3-540-24777-7, March.
    7. Eitan Zemel, 1980. "The Linear Multiple Choice Knapsack Problem," Operations Research, INFORMS, vol. 28(6), pages 1412-1423, December.
    8. Luedtke Alexander R. & van der Laan Mark J., 2016. "Optimal Individualized Treatments in Resource-Limited Settings," The International Journal of Biostatistics, De Gruyter, vol. 12(1), pages 283-303, May.
    9. Gerber, Alan S. & Green, Donald P. & Larimer, Christopher W., 2008. "Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment," American Political Science Review, Cambridge University Press, vol. 102(1), pages 33-48, February.
    10. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael C Knaus, 2022. "Double machine learning-based programme evaluation under unconfoundedness [Econometric methods for program evaluation]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 602-627.
    2. Zequn Jin & Gaoqian Xu & Xi Zheng & Yahong Zhou, 2025. "Policy Learning under Unobserved Confounding: A Robust and Efficient Approach," Papers 2507.20550, arXiv.org.
    3. James Cussens & Julia Hatamyar & Vishalie Shah & Noemi Kreif, 2025. "Fast Learning of Optimal Policy Trees," Papers 2506.15435, arXiv.org.
    4. Patrick Rehill & Nicholas Biddle, 2023. "Transparency challenges in policy evaluation with causal machine learning -- improving usability and accountability," Papers 2310.13240, arXiv.org, revised Mar 2024.
    5. Goller, Daniel & Lechner, Michael & Pongratz, Tamara & Wolff, Joachim, 2025. "Active labor market policies for the long-term unemployed: New evidence from causal machine learning," Labour Economics, Elsevier, vol. 94(C).
    6. Davide Viviano, 2019. "Policy Targeting under Network Interference," Papers 1906.10258, arXiv.org, revised Apr 2024.
    7. Gabriel Okasa, 2022. "Meta-Learners for Estimation of Causal Effects: Finite Sample Cross-Fit Performance," Papers 2201.12692, arXiv.org.
    8. Michael Lechner, 2023. "Causal Machine Learning and its use for public policy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 159(1), pages 1-15, December.
    9. Nathan Kallus, 2023. "Treatment Effect Risk: Bounds and Inference," Management Science, INFORMS, vol. 69(8), pages 4579-4590, August.
    10. Ganesh Karapakula, 2023. "Stable Probability Weighting: Large-Sample and Finite-Sample Estimation and Inference Methods for Heterogeneous Causal Effects of Multivalued Treatments Under Limited Overlap," Papers 2301.05703, arXiv.org, revised Jan 2023.
    11. Achim Ahrens & Alessandra Stampi‐Bombelli & Selina Kurer & Dominik Hangartner, 2024. "Optimal multi‐action treatment allocation: A two‐phase field experiment to boost immigrant naturalization," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(7), pages 1379-1395, November.
    12. Patrick Rehill & Nicholas Biddle, 2025. "Policy Learning for Many Outcomes of Interest: Combining Optimal Policy Trees with Multi-objective Bayesian Optimisation," Computational Economics, Springer;Society for Computational Economics, vol. 66(2), pages 971-1001, August.
    13. Yanqin Fan & Yuan Qi & Gaoqian Xu, 2025. "Policy Learning with $\alpha$-Expected Welfare," Papers 2505.00256, arXiv.org.
    14. Ke Sun & Linglong Kong & Hongtu Zhu & Chengchun Shi, 2024. "ARMA-Design: Optimal Treatment Allocation Strategies for A/B Testing in Partially Observable Time Series Experiments," Papers 2408.05342, arXiv.org, revised Jan 2025.
    15. Patrick Rehill & Nicholas Biddle, 2023. "Fairness Implications of Heterogeneous Treatment Effect Estimation with Machine Learning Methods in Policy-making," Papers 2309.00805, arXiv.org.
    16. Fabian Muny, 2025. "Evaluating Program Sequences with Double Machine Learning: An Application to Labor Market Policies," Papers 2506.11960, arXiv.org.
    17. Asanov, Anastasiya-Mariya & Asanov, Igor & Buenstorf, Guido, 2024. "A low-cost digital first aid tool to reduce psychological distress in refugees: A multi-country randomized controlled trial of self-help online in the first months after the invasion of Ukraine," Social Science & Medicine, Elsevier, vol. 362(C).
    18. Justin Whitehouse & Morgane Austern & Vasilis Syrgkanis, 2025. "Inference on Optimal Policy Values and Other Irregular Functionals via Smoothing," Papers 2507.11780, arXiv.org.
    19. Davide Viviano & Jelena Bradic, 2019. "Synthetic learner: model-free inference on treatments over time," Papers 1904.01490, arXiv.org, revised Aug 2022.
    20. Sung Jae Jun & Sokbae Lee, 2024. "Causal Inference Under Outcome-Based Sampling with Monotonicity Assumptions," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 42(3), pages 998-1009, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2306.11979. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.