IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v42y2022i5p626-636.html
   My bibliography  Save this article

Calculating Expected Value of Sample Information Adjusting for Imperfect Implementation

Author

Listed:
  • Anna Heath

    (Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada
    Division of Biostatistics, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
    Department of Statistical Science, University College London, London, UK)

Abstract

Background The expected value of sample information (EVSI) calculates the value of collecting additional information through a research study with a given design. However, standard EVSI analyses do not account for the slow and often incomplete implementation of the treatment recommendations that follow research. Thus, standard EVSI analyses do not correctly capture the value of the study. Previous research has developed measures to calculate the research value while adjusting for implementation challenges, but estimating these measures is a challenge. Methods Based on a method that assumes the implementation level is related to the strength of evidence in favor of the treatment, 2 implementation-adjusted EVSI calculation methods are developed. These novel methods circumvent the need for analytical calculations, which were restricted to settings in which normality could be assumed. The first method developed in this article uses computationally demanding nested simulations, based on the definition of the implementation-adjusted EVSI. The second method is based on adapting the moment matching method, a recently developed efficient EVSI computation method, to adjust for imperfect implementation. The implementation-adjusted EVSI is then calculated with the 2 methods across 3 examples. Results The maximum difference between the 2 methods is at most 6% in all examples. The efficient computation method is between 6 and 60 times faster than the nested simulation method in this case study and could be used in practice. Conclusions This article permits the calculation of an implementation-adjusted EVSI using realistic assumptions. The efficient estimation method is accurate and can estimate the implementation-adjusted EVSI in practice. By adapting standard EVSI estimation methods, adjustments for imperfect implementation can be made with the same computational cost as a standard EVSI analysis. Highlights Standard expected value of sample information (EVSI) analyses do not account for the fact that treatment implementation following research is often slow and incomplete, meaning they incorrectly capture the value of the study. Two methods, based on nested Monte Carlo sampling and the moment matching EVSI calculation method, are developed to adjust EVSI calculations for imperfect implementation when the speed and level of the implementation of a new treatment depends on the strength of evidence in favor of the treatment. The 2 methods we develop provide similar estimates for the implementation-adjusted EVSI. Our methods extend current EVSI calculation algorithms and thus require limited additional computational complexity.

Suggested Citation

  • Anna Heath, 2022. "Calculating Expected Value of Sample Information Adjusting for Imperfect Implementation," Medical Decision Making, , vol. 42(5), pages 626-636, July.
    • Handle: RePEc:sae:medema:v:42:y:2022:i:5:p:626-636
    DOI: 10.1177/0272989X211073098
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X211073098
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X211073098?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sabine E. Grimm & Simon Dixon & John W. Stevens, 2017. "Assessing the Expected Value of Research Studies in Reducing Uncertainty and Improving Implementation Dynamics," Medical Decision Making, , vol. 37(5), pages 523-533, July.
    2. Stefano Conti & Karl Claxton, 2009. "Dimensions of Design Space: A Decision-Theoretic Approach to Optimal Research Design," Medical Decision Making, , vol. 29(6), pages 643-660, November.
    3. Briggs, Andrew & Sculpher, Mark & Claxton, Karl, 2006. "Decision Modelling for Health Economic Evaluation," OUP Catalogue, Oxford University Press, number 9780198526629.
    4. A. E. Ades & G. Lu & K. Claxton, 2004. "Expected Value of Sample Information Calculations in Medical Decision Modeling," Medical Decision Making, , vol. 24(2), pages 207-227, March.
    5. Claxton, Karl, 1999. "The irrelevance of inference: a decision-making approach to the stochastic evaluation of health care technologies," Journal of Health Economics, Elsevier, vol. 18(3), pages 341-364, June.
    6. Simon Eckermann, 2017. "Health Economics from Theory to Practice," Springer Books, Springer, number 978-3-319-50613-5, September.
    7. Anna Heath & Ioanna Manolopoulou & Gianluca Baio, 2019. "Estimating the Expected Value of Sample Information across Different Sample Sizes Using Moment Matching and Nonlinear Regression," Medical Decision Making, , vol. 39(4), pages 347-359, May.
    8. Anna Heath & Natalia Kunst & Christopher Jackson & Mark Strong & Fernando Alarid-Escudero & Jeremy D. Goldhaber-Fiebert & Gianluca Baio & Nicolas A. Menzies & Hawre Jalal, 2020. "Calculating the Expected Value of Sample Information in Practice: Considerations from 3 Case Studies," Medical Decision Making, , vol. 40(3), pages 314-326, April.
    9. Hawre Jalal & Jeremy D. Goldhaber-Fiebert & Karen M. Kuntz, 2015. "Computing Expected Value of Partial Sample Information from Probabilistic Sensitivity Analysis Using Linear Regression Metamodeling," Medical Decision Making, , vol. 35(5), pages 584-595, July.
    10. Anna Heath & Ioanna Manolopoulou & Gianluca Baio, 2018. "Efficient Monte Carlo Estimation of the Expected Value of Sample Information Using Moment Matching," Medical Decision Making, , vol. 38(2), pages 163-173, February.
    11. Elisabeth Fenwick & Karl Claxton & Mark Sculpher, 2008. "The Value of Implementation and the Value of Information: Combined and Uneven Development," Medical Decision Making, , vol. 28(1), pages 21-32, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anna Heath & Mark Strong & David Glynn & Natalia Kunst & Nicky J. Welton & Jeremy D. Goldhaber-Fiebert, 2022. "Simulating Study Data to Support Expected Value of Sample Information Calculations: A Tutorial," Medical Decision Making, , vol. 42(2), pages 143-155, February.
    2. Mathyn Vervaart & Mark Strong & Karl P. Claxton & Nicky J. Welton & Torbjørn Wisløff & Eline Aas, 2022. "An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial," Medical Decision Making, , vol. 42(5), pages 612-625, July.
    3. Wei Fang & Zhenru Wang & Michael B. Giles & Chris H. Jackson & Nicky J. Welton & Christophe Andrieu & Howard Thom, 2022. "Multilevel and Quasi Monte Carlo Methods for the Calculation of the Expected Value of Partial Perfect Information," Medical Decision Making, , vol. 42(2), pages 168-181, February.
    4. Anna Heath & Petros Pechlivanoglou, 2022. "Prioritizing Research in an Era of Personalized Medicine: The Potential Value of Unexplained Heterogeneity," Medical Decision Making, , vol. 42(5), pages 649-660, July.
    5. WH Rogowski, 2013. "An Economic Theory Of The Fourth Hurdle," Health Economics, John Wiley & Sons, Ltd., vol. 22(5), pages 600-610, May.
    6. Mathyn Vervaart & Eline Aas & Karl P. Claxton & Mark Strong & Nicky J. Welton & Torbjørn Wisløff & Anna Heath, 2023. "General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations," Medical Decision Making, , vol. 43(5), pages 595-609, July.
    7. Andrija S Grustam & Nasuh Buyukkaramikli & Ron Koymans & Hubertus J M Vrijhoef & Johan L Severens, 2019. "Value of information analysis in telehealth for chronic heart failure management," PLOS ONE, Public Library of Science, vol. 14(6), pages 1-23, June.
    8. Daniele Bregantini, 2014. "Don’t Stop ’Til You Get Enough: a quickest detection approach to HTA," Discussion Papers 14/04, Department of Economics, University of York.
    9. Hester V Eeren & Saskia J Schawo & Ron H J Scholte & Jan J V Busschbach & Leona Hakkaart, 2015. "Value of Information Analysis Applied to the Economic Evaluation of Interventions Aimed at Reducing Juvenile Delinquency: An Illustration," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-15, July.
    10. Lauren E. Cipriano & Thomas A. Weber, 2018. "Population-level intervention and information collection in dynamic healthcare policy," Health Care Management Science, Springer, vol. 21(4), pages 604-631, December.
    11. Qi Cao & Erik Buskens & Hans L. Hillege & Tiny Jaarsma & Maarten Postma & Douwe Postmus, 2019. "Stratified treatment recommendation or one-size-fits-all? A health economic insight based on graphical exploration," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(3), pages 475-482, April.
    12. Isaac Corro Ramos & Maureen P. M. H. Rutten-van Mölken & Maiwenn J. Al, 2013. "The Role of Value-of-Information Analysis in a Health Care Research Priority Setting," Medical Decision Making, , vol. 33(4), pages 472-489, May.
    13. McKenna, Claire & Chalabi, Zaid & Epstein, David & Claxton, Karl, 2010. "Budgetary policies and available actions: A generalisation of decision rules for allocation and research decisions," Journal of Health Economics, Elsevier, vol. 29(1), pages 170-181, January.
    14. Rachael L. Fleurence, 2007. "Setting priorities for research: a practical application of 'payback' and expected value of information," Health Economics, John Wiley & Sons, Ltd., vol. 16(12), pages 1345-1357.
    15. Claire McKenna & Karl Claxton, 2011. "Addressing Adoption and Research Design Decisions Simultaneously," Medical Decision Making, , vol. 31(6), pages 853-865, November.
    16. Stefano Conti & Karl Claxton, 2008. "Dimensions of design space: a decision-theoretic approach to optimal research design," Working Papers 038cherp, Centre for Health Economics, University of York.
    17. Haitham Tuffaha & Claire Rothery & Natalia Kunst & Chris Jackson & Mark Strong & Stephen Birch, 2021. "A Review of Web-Based Tools for Value-of-Information Analysis," Applied Health Economics and Health Policy, Springer, vol. 19(5), pages 645-651, September.
    18. Anirban Basu & David Meltzer, 2012. "Private Manufacturers’ Thresholds to Invest in Comparative Effectiveness Trials," PharmacoEconomics, Springer, vol. 30(10), pages 859-868, October.
    19. Dirk Müller & Eleanor Pullenayegum & Afschin Gandjour, 2015. "Impact of small study bias on cost-effectiveness acceptability curves and value of information analyses," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(2), pages 219-223, March.
    20. Kasper M. Johannesen & Karl Claxton & Mark J. Sculpher & Allan J. Wailoo, 2018. "How to design the cost‐effectiveness appraisal process of new healthcare technologies to maximise population health: A conceptual framework," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 41-54, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:42:y:2022:i:5:p:626-636. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.