IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v79y2017i5p1439-1462.html
   My bibliography  Save this article

A Bayesian decision theoretic model of sequential experimentation with delayed response

Author

Listed:
  • Stephen Chick
  • Martin Forster
  • Paolo Pertile

Abstract

We solve a Bayesian decision-theoretic model of a sequential experiment in which the real-valued primary end point is observed with delay. The solution yields a unified policy defining the optimal 'do notexperiment'/'fixed sample size experiment'/'sequential experiment' regions as a function of the prior mean. The model can value the expected benefits accruing to study units, the fixed costs of switching from control to treatment, and allows the number of study units to benefit from a stopping decision to fall as the number of study units recruited to the experiment rises. We apply the model to the field of medical statistics, using data from a published trial investigating the clinical- and cost-effectiveness of drug-eluting stents versus bare metal stents. We demonstrate the model’s superiority over alternative trial designs when judged according to the maximisation of the net benefits of the trial, minus sampling costs, and we investigate how the size of the delay determines the optimal choice of trial design. The optimal policy also performs well when judged according to the probability of making the correct selection of health technology.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Stephen Chick & Martin Forster & Paolo Pertile, 2017. "A Bayesian decision theoretic model of sequential experimentation with delayed response," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(5), pages 1439-1462, November.
  • Handle: RePEc:bla:jorssb:v:79:y:2017:i:5:p:1439-1462
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/rssb.12225
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Felipe Caro & Jérémie Gallien, 2007. "Dynamic Assortment with Demand Learning for Seasonal Consumer Goods," Management Science, INFORMS, vol. 53(2), pages 276-292, February.
    2. Peter I. Frazier & Warren B. Powell, 2010. "Paradoxes in Learning and the Marginal Value of Information," Decision Analysis, INFORMS, vol. 7(4), pages 378-403, December.
    3. Dimitris Bertsimas & Adam J. Mersereau, 2007. "A Learning Approach for Interactive Marketing to a Customer Segment," Operations Research, INFORMS, vol. 55(6), pages 1120-1135, December.
    4. Ahuja, Vishal & Birge, John R., 2016. "Response-adaptive designs for clinical trials: Simultaneous learning from multiple patients," European Journal of Operational Research, Elsevier, vol. 248(2), pages 619-633.
    5. Lisa V. Hampson & Christopher Jennison, 2013. "Group sequential tests for delayed responses (with discussion)," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(1), pages 3-54, January.
    6. Brezzi, Monica & Lai, Tze Leung, 2002. "Optimal learning and experimentation in bandit problems," Journal of Economic Dynamics and Control, Elsevier, vol. 27(1), pages 87-108, November.
    7. S. A. Murphy, 2003. "Optimal dynamic treatment regimes," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 65(2), pages 331-355, May.
    8. Paolo Pertile & Martin Forster & Davide La Torre, 2010. "Optimal sequential sampling rules for the economic evaluation of health technologies," Discussion Papers 10/24, Department of Economics, University of York.
    9. Stephen E. Chick & Noah Gans, 2009. "Economic Analysis of Simulation Selection Problems," Management Science, INFORMS, vol. 55(3), pages 421-437, March.
    10. Marina Roshini Sooriyarachchi & John Whitehead & Kim Bolland & Anne Whitehead, 2003. "Incorporating Data Received after a Sequential Trial Has Stopped into the Final Analysis: Implementation and Comparison of Methods," Biometrics, The International Biometric Society, vol. 59(3), pages 701-709, September.
    11. Jennifer L. Kirk & Michael P. Fay, 2014. "An Introduction to Practical Sequential Inferences via Single-Arm Binary Response Studies Using the binseqtest R Package," The American Statistician, Taylor & Francis Journals, vol. 68(4), pages 230-242, November.
    12. Stephen E. Chick & Peter Frazier, 2012. "Sequential Sampling with Economics of Selection Procedures," Management Science, INFORMS, vol. 58(3), pages 550-569, March.
    13. Alessandro Arlotto & Stephen E. Chick & Noah Gans, 2014. "Optimal Hiring and Retention Policies for Heterogeneous Workers Who Learn," Management Science, INFORMS, vol. 60(1), pages 110-129, January.
    14. Paolo Pertile & Martin Forster & Davide La Torre, 2014. "Optimal Bayesian sequential sampling rules for the economic evaluation of health technologies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 177(2), pages 419-438, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nikhil Bhat & Vivek F. Farias & Ciamac C. Moallemi & Deeksha Sinha, 2020. "Near-Optimal A-B Testing," Management Science, INFORMS, vol. 66(10), pages 4477-4495, October.
    2. Andres Alban & Stephen E. Chick & Martin Forster, 2020. "Value-based clinical trials: selecting trial lengths and recruitment rates in different regulatory contexts," Discussion Papers 20/01, Department of Economics, University of York.
    3. Thijssen, Jacco J.J. & Bregantini, Daniele, 2017. "Costly sequential experimentation and project valuation with an application to health technology assessment," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 202-229.
    4. Williamson, S. Faye & Jacko, Peter & Jaki, Thomas, 2022. "Generalisations of a Bayesian decision-theoretic randomisation procedure and the impact of delayed responses," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    5. Panos Kouvelis & Joseph Milner & Zhili Tian, 2017. "Clinical Trials for New Drug Development: Optimal Investment and Application," Manufacturing & Service Operations Management, INFORMS, vol. 19(3), pages 437-452, July.
    6. Vishal Ahuja & John R. Birge, 2020. "An Approximation Approach for Response-Adaptive Clinical Trial Design," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 877-894, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eric M. Schwartz & Eric T. Bradlow & Peter S. Fader, 2017. "Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments," Marketing Science, INFORMS, vol. 36(4), pages 500-522, July.
    2. Raluca M. Ursu & Qingliang Wang & Pradeep K. Chintagunta, 2020. "Search Duration," Marketing Science, INFORMS, vol. 39(5), pages 849-871, September.
    3. Stephen E. Chick & Peter Frazier, 2012. "Sequential Sampling with Economics of Selection Procedures," Management Science, INFORMS, vol. 58(3), pages 550-569, March.
    4. Vishal Ahuja & John R. Birge, 2020. "An Approximation Approach for Response-Adaptive Clinical Trial Design," INFORMS Journal on Computing, INFORMS, vol. 32(4), pages 877-894, October.
    5. Jing Xie & Peter I. Frazier, 2013. "Sequential Bayes-Optimal Policies for Multiple Comparisons with a Known Standard," Operations Research, INFORMS, vol. 61(5), pages 1174-1189, October.
    6. Francisco Alvarez, 2018. "Decomposing risk in an exploitation–exploration problem with endogenous termination time," Annals of Operations Research, Springer, vol. 261(1), pages 45-77, February.
    7. Thijssen, Jacco J.J. & Bregantini, Daniele, 2017. "Costly sequential experimentation and project valuation with an application to health technology assessment," Journal of Economic Dynamics and Control, Elsevier, vol. 77(C), pages 202-229.
    8. Li Chen & Adam J.Mersereau & Zhe (Frank) Wang, 2017. "Optimal Merchandise Testing with Limited Inventory," Operations Research, INFORMS, vol. 65(4), pages 968-991, August.
    9. Jacko, Peter & Niño Mora, José, 2009. "An index for dynamic product promotion and the knapsack problem for perishable items," DES - Working Papers. Statistics and Econometrics. WS ws093111, Universidad Carlos III de Madrid. Departamento de Estadística.
    10. Nikhil Bhat & Vivek F. Farias & Ciamac C. Moallemi & Deeksha Sinha, 2020. "Near-Optimal A-B Testing," Management Science, INFORMS, vol. 66(10), pages 4477-4495, October.
    11. Williamson, S. Faye & Jacko, Peter & Jaki, Thomas, 2022. "Generalisations of a Bayesian decision-theoretic randomisation procedure and the impact of delayed responses," Computational Statistics & Data Analysis, Elsevier, vol. 174(C).
    12. Elea McDonnell Feit & Ron Berman, 2019. "Test & Roll: Profit-Maximizing A/B Tests," Marketing Science, INFORMS, vol. 38(6), pages 1038-1058, November.
    13. Martin Forster & Paolo Pertile, 2013. "Optimal decision rules for HTA under uncertainty: a wider, dynamic perspective," Health Economics, John Wiley & Sons, Ltd., vol. 22(12), pages 1507-1514, December.
    14. Onesun Steve Yoo & Kevin McCardle, 2020. "The Valuator’s Curse: Decision Analysis of Overvaluation and Disappointment in Acquisition," Decision Analysis, INFORMS, vol. 17(4), pages 299-313, December.
    15. Vivek F. Farias & Ritesh Madan, 2011. "The Irrevocable Multiarmed Bandit Problem," Operations Research, INFORMS, vol. 59(2), pages 383-399, April.
    16. Kevin Glazebrook & Joern Meissner & Jochen Schurr, 2012. "How big should my store be? On the interplay between shelf-space, demand learning and assortment decisions," Working Papers MRG/0021, Department of Management Science, Lancaster University, revised Dec 2012.
    17. Ahuja, Vishal & Birge, John R., 2016. "Response-adaptive designs for clinical trials: Simultaneous learning from multiple patients," European Journal of Operational Research, Elsevier, vol. 248(2), pages 619-633.
    18. Jun Luo & L. Jeff Hong & Barry L. Nelson & Yang Wu, 2015. "Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments," Operations Research, INFORMS, vol. 63(5), pages 1177-1194, October.
    19. Daniel Russo, 2020. "Simple Bayesian Algorithms for Best-Arm Identification," Operations Research, INFORMS, vol. 68(6), pages 1625-1647, November.
    20. Diana M. Negoescu & Kostas Bimpikis & Margaret L. Brandeau & Dan A. Iancu, 2018. "Dynamic Learning of Patient Response Types: An Application to Treating Chronic Diseases," Management Science, INFORMS, vol. 64(8), pages 3469-3488, August.

    More about this item

    JEL classification:

    • C11 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Bayesian Analysis: General
    • C61 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Optimization Techniques; Programming Models; Dynamic Analysis
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • I10 - Health, Education, and Welfare - - Health - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:79:y:2017:i:5:p:1439-1462. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.