IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2210.15841.html
   My bibliography  Save this paper

How to sample and when to stop sampling: The generalized Wald problem and minimax policies

Author

Listed:
  • Karun Adusumilli

Abstract

The aim of this paper is to develop techniques for incorporating the cost of information into experimental design. Specifically, we study sequential experiments where sampling is costly and a decision-maker aims to determine the best treatment for full scale implementation by (1) adaptively allocating units to two possible treatments, and (2) stopping the experiment when the expected welfare (inclusive of sampling costs) from implementing the chosen treatment is maximized. Working under the diffusion limit, we describe the optimal policies under the minimax regret criterion. Under small cost asymptotics, the same policies are also optimal under parametric and non-parametric distributions of outcomes. The minimax optimal sampling rule is just the Neyman allocation; it is independent of sampling costs and does not adapt to previous outcomes. The decision-maker stops sampling when the average difference between the treatment outcomes, multiplied by the number of observations collected until that point, exceeds a specific threshold. The results derived here also apply to best arm identification with two arms.

Suggested Citation

  • Karun Adusumilli, 2022. "How to sample and when to stop sampling: The generalized Wald problem and minimax policies," Papers 2210.15841, arXiv.org, revised Feb 2024.
    • Handle: RePEc:arx:papers:2210.15841
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2210.15841
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Drew Fudenberg & Philipp Strack & Tomasz Strzalecki, 2018. "Speed, Accuracy, and the Optimal Timing of Choices," American Economic Review, American Economic Association, vol. 108(12), pages 3651-3684, December.
    2. Jimmy Chan & Alessandro Lizzeri & Wing Suen & Leeat Yariv, 2018. "Deliberating Collective Decisions," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 85(2), pages 929-963.
    3. Keisuke Hirano & Jack R. Porter, 2009. "Asymptotics for Statistical Treatment Rules," Econometrica, Econometric Society, vol. 77(5), pages 1683-1701, September.
    4. Annie Liang & Xiaosheng Mu & Vasilis Syrgkanis, 2022. "Dynamically Aggregating Diverse Information," Econometrica, Econometric Society, vol. 90(1), pages 47-80, January.
    5. Sims, Christopher A., 2003. "Implications of rational inattention," Journal of Monetary Economics, Elsevier, vol. 50(3), pages 665-690, April.
    6. Charles F. Manski, 2021. "Econometrics for Decision Making: Building Foundations Sketched by Haavelmo and Wald," Econometrica, Econometric Society, vol. 89(6), pages 2827-2853, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chao Qin & Daniel Russo, 2024. "Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification," Papers 2402.10592, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Philippe Jehiel & Jakub Steiner, 2020. "Selective Sampling with Information-Storage Constraints [On interim rationality, belief formation and learning in decision problems with bounded memory]," The Economic Journal, Royal Economic Society, vol. 130(630), pages 1753-1781.
    2. Manski, Charles F., 2023. "Probabilistic prediction for binary treatment choice: With focus on personalized medicine," Journal of Econometrics, Elsevier, vol. 234(2), pages 647-663.
    3. Pëllumb Reshidi & Alessandro Lizzeri & Leeat Yariv & Jimmy Chan & Wing Suen, 2021. "Individual and Collective Information Acquisition: An Experimental Study," CESifo Working Paper Series 9468, CESifo.
    4. Stephanie M. Smith & Ian Krajbich & Ryan Webb, 2019. "Estimating the dynamic role of attention via random utility," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 5(1), pages 97-111, August.
    5. Jakub Steiner & Colin Stewart & Filip Matějka, 2017. "Rational Inattention Dynamics: Inertia and Delay in Decision‐Making," Econometrica, Econometric Society, vol. 85, pages 521-553, March.
    6. Xavier Gabaix, 2017. "Behavioral Inattention," NBER Working Papers 24096, National Bureau of Economic Research, Inc.
    7. Timothy Christensen & Hyungsik Roger Moon & Frank Schorfheide, 2022. "Optimal Decision Rules when Payoffs are Partially Identified," Papers 2204.11748, arXiv.org, revised May 2023.
    8. Pëllumb Reshidi & Alessandro Lizzeri & Leeat Yariv & Jimmy Chan & Wing Suen, 2022. "Individual and Collective Information Acquisition: An Experimental Study," Working Papers 312, Princeton University, Department of Economics, Center for Economic Policy Studies..
    9. Yeon-Koo Che & Konrad Mierendorff, 2019. "Optimal Dynamic Allocation of Attention," American Economic Review, American Economic Association, vol. 109(8), pages 2993-3029, August.
    10. Maximilian Blesch & Philipp Eisenhauer, 2023. "Robust Decision-Making under Risk and Ambiguity," Rationality and Competition Discussion Paper Series 463, CRC TRR 190 Rationality and Competition.
    11. Emeric Henry & Marco Loseto & Marco Ottaviani, 2022. "Regulation with Experimentation: Ex Ante Approval, Ex Post Withdrawal, and Liability," Management Science, INFORMS, vol. 68(7), pages 5330-5347, July.
    12. Charles F. Manski & Aleksey Tetenov, 2023. "Statistical decision theory respecting stochastic dominance," The Japanese Economic Review, Springer, vol. 74(4), pages 447-469, October.
    13. Seungjin Han & Julius Owusu & Youngki Shin, 2022. "Statistical Treatment Rules under Social Interaction," Papers 2209.09077, arXiv.org, revised Nov 2022.
    14. Philippas, Dionisis & Dragomirescu-Gaina, Catalin & Goutte, Stéphane & Nguyen, Duc Khuong, 2021. "Investors’ attention and information losses under market stress," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 1112-1127.
    15. Toru Kitagawa & Sokbae Lee & Chen Qiu, 2022. "Treatment Choice with Nonlinear Regret," Papers 2205.08586, arXiv.org, revised Feb 2024.
    16. Martin, Daniel & Muñoz-Rodriguez, Edwin, 2022. "Cognitive costs and misperceived incentives: Evidence from the BDM mechanism," European Economic Review, Elsevier, vol. 148(C).
    17. Kaido, Hiroaki, 2017. "Asymptotically Efficient Estimation Of Weighted Average Derivatives With An Interval Censored Variable," Econometric Theory, Cambridge University Press, vol. 33(5), pages 1218-1241, October.
    18. Persson, Petra, 2018. "Attention manipulation and information overload," Behavioural Public Policy, Cambridge University Press, vol. 2(1), pages 78-106, May.
    19. Kurt Lewis, 2009. "The Two-Period Rational Inattention Model: Accelerations and Analyses," Computational Economics, Springer;Society for Computational Economics, vol. 33(1), pages 79-97, February.
    20. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP72/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2210.15841. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.