IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v79y2017i5p1439-1462.html
   My bibliography  Save this article

A Bayesian decision theoretic model of sequential experimentation with delayed response

Author

Listed:
  • Stephen Chick
  • Martin Forster
  • Paolo Pertile

Abstract

We solve a Bayesian decision-theoretic model of a sequential experiment in which the real-valued primary end point is observed with delay. The solution yields a unified policy defining the optimal 'do notexperiment'/'fixed sample size experiment'/'sequential experiment' regions as a function of the prior mean. The model can value the expected benefits accruing to study units, the fixed costs of switching from control to treatment, and allows the number of study units to benefit from a stopping decision to fall as the number of study units recruited to the experiment rises. We apply the model to the field of medical statistics, using data from a published trial investigating the clinical- and cost-effectiveness of drug-eluting stents versus bare metal stents. We demonstrate the model’s superiority over alternative trial designs when judged according to the maximisation of the net benefits of the trial, minus sampling costs, and we investigate how the size of the delay determines the optimal choice of trial design. The optimal policy also performs well when judged according to the probability of making the correct selection of health technology.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Stephen Chick & Martin Forster & Paolo Pertile, 2017. "A Bayesian decision theoretic model of sequential experimentation with delayed response," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1439-1462, November.
    • Handle: RePEc:bla:jorssb:v:79:y:2017:i:5:p:1439-1462
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/rssb.12225
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Peter I. Frazier & Warren B. Powell, 2010. "Paradoxes in Learning and the Marginal Value of Information," Decision Analysis, INFORMS, vol. 7(4), pages 378-403, December.
    2. Lisa V. Hampson & Christopher Jennison, 2013. "Group sequential tests for delayed responses (with discussion)," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(1), pages 3-54, January.
    3. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    4. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    5. Marina Roshini Sooriyarachchi & John Whitehead & Kim Bolland & Anne Whitehead, 2003. "Incorporating Data Received after a Sequential Trial Has Stopped into the Final Analysis: Implementation and Comparison of Methods," Biometrics, The International Biometric Society, vol. 59(3), pages 701-709, September.
    6. Alessandro Arlotto & Stephen E. Chick & Noah Gans, 2014. "Optimal Hiring and Retention Policies for Heterogeneous Workers Who Learn," Management Science, INFORMS, vol. 60(1), pages 110-129, January.
    7. Paolo Pertile & Martin Forster & Davide La Torre, 2014. "Optimal Bayesian sequential sampling rules for the economic evaluation of health technologies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 177(2), pages 419-438, February.
    8. Felipe Caro & Jérémie Gallien, 2007. "Dynamic Assortment with Demand Learning for Seasonal Consumer Goods," Management Science, INFORMS, vol. 53(2), pages 276-292, February.
    9. Dimitris Bertsimas & Adam J. Mersereau, 2007. "A Learning Approach for Interactive Marketing to a Customer Segment," Operations Research, INFORMS, vol. 55(6), pages 1120-1135, December.
    10. Ahuja, Vishal & Birge, John R., 2016. "Response-adaptive designs for clinical trials: Simultaneous learning from multiple patients," European Journal of Operational Research, Elsevier, vol. 248(2), pages 619-633.
    11. Brezzi, Monica & Lai, Tze Leung, 2002. "Optimal learning and experimentation in bandit problems," Journal of Economic Dynamics and Control, Elsevier, vol. 27(1), pages 87-108, November.
    12. Paolo Pertile & Martin Forster & Davide La Torre, 2010. "Optimal sequential sampling rules for the economic evaluation of health technologies," Discussion Papers 10/24, Department of Economics, University of York.
    13. Jennifer L. Kirk & Michael P. Fay, 2014. "An Introduction to Practical Sequential Inferences via Single-Arm Binary Response Studies Using the binseqtest R Package," The American Statistician, Taylor & Francis Journals, vol. 68(4), pages 230-242, November.
    14. Stephen E. Chick & Peter Frazier, 2012. "Sequential Sampling with Economics of Selection Procedures," Management Science, INFORMS, vol. 58(3), pages 550-569, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Amir Ali Nasrollahzadeh & Amin Khademi, 2022. "Dynamic Programming for Response-Adaptive Dose-Finding Clinical Trials," INFORMS Journal on Computing, INFORMS, vol. 34(2), pages 1176-1190, March.
    2. Nikhil Bhat & Vivek F. Farias & Ciamac C. Moallemi & Deeksha Sinha, 2020. "Near-Optimal A-B Testing," Management Science, INFORMS, vol. 66(10), pages 4477-4495, October.
    3. Arielle Anderer & Hamsa Bastani & John Silberholz, 2022. "Adaptive Clinical Trial Designs with Surrogates: When Should We Bother?," Management Science, INFORMS, vol. 68(3), pages 1982-2002, March.
    4. Andres Alban & Stephen E. Chick & Martin Forster, 2023. "Value-Based Clinical Trials: Selecting Recruitment Rates and Trial Lengths in Different Regulatory Contexts," Management Science, INFORMS, vol. 69(6), pages 3516-3535, June.
    5. Andres Alban & Stephen E. Chick & Martin Forster, 2020. "Value-based clinical trials: selecting trial lengths and recruitment rates in different regulatory contexts," Discussion Papers 20/01, Department of Economics, University of York.
    6. Thijssen, Jacco J.J. & Bregantini, Daniele, 2017. "Costly sequential experimentation and project valuation with an application to health technology assessment," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 202-229.
    7. Williamson, S. Faye & Jacko, Peter & Jaki, Thomas, 2022. "Generalisations of a Bayesian decision-theoretic randomisation procedure and the impact of delayed responses," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    8. Panos Kouvelis & Joseph Milner & Zhili Tian, 2017. "Clinical Trials for New Drug Development: Optimal Investment and Application," Manufacturing & Service Operations Management, INFORMS, vol. 19(3), pages 437-452, July.
    9. Vishal Ahuja & John R. Birge, 2020. "An Approximation Approach for Response-Adaptive Clinical Trial Design," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 877-894, October.
    10. Stephen E. Chick & Noah Gans & Özge Yapar, 2022. "Bayesian Sequential Learning for Clinical Trials of Multiple Correlated Medical Interventions," Management Science, INFORMS, vol. 68(7), pages 4919-4938, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Victor F. Araman & René A. Caldentey, 2022. "Diffusion Approximations for a Class of Sequential Experimentation Problems," Management Science, INFORMS, vol. 68(8), pages 5958-5979, August.
    2. Stephen E. Chick & Noah Gans & Özge Yapar, 2022. "Bayesian Sequential Learning for Clinical Trials of Multiple Correlated Medical Interventions," Management Science, INFORMS, vol. 68(7), pages 4919-4938, July.
    3. Eric M. Schwartz & Eric T. Bradlow & Peter S. Fader, 2017. "Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments," Marketing Science, INFORMS, vol. 36(4), pages 500-522, July.
    4. Raluca M. Ursu & Qingliang Wang & Pradeep K. Chintagunta, 2020. "Search Duration," Marketing Science, INFORMS, vol. 39(5), pages 849-871, September.
    5. Stephen E. Chick & Peter Frazier, 2012. "Sequential Sampling with Economics of Selection Procedures," Management Science, INFORMS, vol. 58(3), pages 550-569, March.
    6. Vishal Ahuja & John R. Birge, 2020. "An Approximation Approach for Response-Adaptive Clinical Trial Design," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 877-894, October.
    7. Jing Xie & Peter I. Frazier, 2013. "Sequential Bayes-Optimal Policies for Multiple Comparisons with a Known Standard," Operations Research, INFORMS, vol. 61(5), pages 1174-1189, October.
    8. Andres Alban & Stephen E. Chick & Martin Forster, 2023. "Value-Based Clinical Trials: Selecting Recruitment Rates and Trial Lengths in Different Regulatory Contexts," Management Science, INFORMS, vol. 69(6), pages 3516-3535, June.
    9. Francisco Alvarez, 2018. "Decomposing risk in an exploitation–exploration problem with endogenous termination time," Annals of Operations Research, Springer, vol. 261(1), pages 45-77, February.
    10. Thijssen, Jacco J.J. & Bregantini, Daniele, 2017. "Costly sequential experimentation and project valuation with an application to health technology assessment," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 202-229.
    11. Santiago R. Balseiro & David B. Brown & Chen Chen, 2021. "Dynamic Pricing of Relocating Resources in Large Networks," Management Science, INFORMS, vol. 67(7), pages 4075-4094, July.
    12. Chao Qin & Daniel Russo, 2024. "Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification," Papers 2402.10592, arXiv.org.
    13. Li Chen & Adam J.Mersereau & Zhe (Frank) Wang, 2017. "Optimal Merchandise Testing with Limited Inventory," Operations Research, INFORMS, vol. 65(4), pages 968-991, August.
    14. Haihui Shen & L. Jeff Hong & Xiaowei Zhang, 2021. "Ranking and Selection with Covariates for Personalized Decision Making," INFORMS Journal on Computing, INFORMS, vol. 33(4), pages 1500-1519, October.
    15. Jacko, Peter & Niño Mora, José, 2009. "An index for dynamic product promotion and the knapsack problem for perishable items," DES - Working Papers. Statistics and Econometrics. WS ws093111, Universidad Carlos III de Madrid. Departamento de Estadística.
    16. Arielle Anderer & Hamsa Bastani & John Silberholz, 2022. "Adaptive Clinical Trial Designs with Surrogates: When Should We Bother?," Management Science, INFORMS, vol. 68(3), pages 1982-2002, March.
    17. Nikhil Bhat & Vivek F. Farias & Ciamac C. Moallemi & Deeksha Sinha, 2020. "Near-Optimal A-B Testing," Management Science, INFORMS, vol. 66(10), pages 4477-4495, October.
    18. Williamson, S. Faye & Jacko, Peter & Jaki, Thomas, 2022. "Generalisations of a Bayesian decision-theoretic randomisation procedure and the impact of delayed responses," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    19. Elea McDonnell Feit & Ron Berman, 2019. "Test & Roll: Profit-Maximizing A/B Tests," Marketing Science, INFORMS, vol. 38(6), pages 1038-1058, November.
    20. Martin Forster & Paolo Pertile, 2013. "Optimal decision rules for HTA under uncertainty: a wider, dynamic perspective," Health Economics, John Wiley & Sons, Ltd., vol. 22(12), pages 1507-1514, December.

    More about this item

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • I10 - Health, Education, and Welfare - - Health - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:79:y:2017:i:5:p:1439-1462. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.