IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v45y2025i2p127-142.html

Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent

Author

Listed:
  • Jeremy D. Goldhaber-Fiebert

    (Department of Health Policy, Stanford School of Medicine, Stanford, CA, USA
    Center for Health Policy, Freeman Spogli Institute, Stanford University, Stanford, CA, USA)

  • Hawre Jalal

    (School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada)

  • Fernando Alarid-Escudero

    (Department of Health Policy, Stanford School of Medicine, Stanford, CA, USA
    Center for Health Policy, Freeman Spogli Institute, Stanford University, Stanford, CA, USA)

Abstract

Purpose Individual-level state-transition microsimulations (iSTMs) have proliferated for economic evaluations in place of cohort state transition models (cSTMs). Probabilistic economic evaluations quantify decision uncertainty and value of information (VOI). Previous studies show that iSTMs provide unbiased estimates of expected incremental net monetary benefits (EINMB), but statistical properties of iSTM-produced estimates of decision uncertainty and VOI remain uncharacterized. Methods We compare iSTM-produced estimates of decision uncertainty and VOI to corresponding cSTMs. For a 2-alternative decision and normally distributed incremental costs and benefits, we derive analytical expressions for the probability of being cost-effective and the expected value of perfect information (EVPI) for cSTMs and iSTMs, accounting for correlations in incremental outcomes at the population and individual levels. We use numerical simulations to illustrate our findings and explore the impact of relaxing normality assumptions or having >2 decision alternatives. Results iSTM estimates of decision uncertainty and VOI are biased but asymptotically consistent (i.e., bias approaches 0 as number of microsimulated individuals approaches infinity). Decision uncertainty depends on 1 tail of the INMB distribution (e.g., P[INMB

Suggested Citation

  • Jeremy D. Goldhaber-Fiebert & Hawre Jalal & Fernando Alarid-Escudero, 2025. "Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent," Medical Decision Making, , vol. 45(2), pages 127-142, February.
    • Handle: RePEc:sae:medema:v:45:y:2025:i:2:p:127-142
    DOI: 10.1177/0272989X241305414
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X241305414
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0272989X241305414?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023, October.
    2. Anirban Basu & David Meltzer, 2007. "Value of Information on Preference Heterogeneity and Individualized Care," Medical Decision Making, , vol. 27(2), pages 112-127, March.
    3. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023.
    4. Hawre Jalal & Jeremy D. Goldhaber-Fiebert & Karen M. Kuntz, 2015. "Computing Expected Value of Partial Sample Information from Probabilistic Sensitivity Analysis Using Linear Regression Metamodeling," Medical Decision Making, , vol. 35(5), pages 584-595, July.
    5. Stefano Conti & Karl Claxton, 2009. "Dimensions of Design Space: A Decision-Theoretic Approach to Optimal Research Design," Medical Decision Making, , vol. 29(6), pages 643-660, November.
    6. Tae Yoon Lee & Paul Gustafson & Mohsen Sadatsafavi, 2023. "Closed-Form Solution of the Unit Normal Loss Integral in 2 Dimensions, with Application in Value-of-Information Analysis," Medical Decision Making, , vol. 43(5), pages 621-626, July.
    7. Stavroula A. Chrysanthopoulou & Carolyn M. Rutter & Constantine A. Gatsonis, 2021. "Bayesian versus Empirical Calibration of Microsimulation Models: A Comparative Analysis," Medical Decision Making, , vol. 41(6), pages 714-726, August.
    8. Fernando Alarid-Escudero & Richard F. MacLehose & Yadira Peralta & Karen M. Kuntz & Eva A. Enns, 2018. "Nonidentifiability in Model Calibration and Implications for Medical Decision Making," Medical Decision Making, , vol. 38(7), pages 810-821, October.
    9. Eline M. Krijkamp & Fernando Alarid-Escudero & Eva A. Enns & Hawre J. Jalal & M. G. Myriam Hunink & Petros Pechlivanoglou, 2018. "Microsimulation Modeling for Health Decision Sciences Using R: A Tutorial," Medical Decision Making, , vol. 38(3), pages 400-422, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Koen Degeling & Jonathan Karnon & Michiel Ven & Alan Brennan & Hendrik Koffijberg, 2025. "Discrete Event Simulation in R using the ‘Simmer’ Package for Health Economic Modelling: A Tutorial and Illustration in Colon Cancer," Applied Health Economics and Health Policy, Springer, vol. 23(6), pages 961-975, November.
    2. Adam Irving & Dennis Petrie & Anthony Harris & Laura Fanning & Erica M. Wood & Elizabeth Moore & Cameron Wellard & Neil Waters & Bradley Augustson & Gordon Cook & Francesca Gay & Georgia McCaughan & P, 2026. "A Post-Market Economic Evaluation of Bortezomib, Lenalidomide and Dexamethasone Versus Pre-funding Standard of Care for Newly Diagnosed Multiple Myeloma Using Registry Data," PharmacoEconomics, Springer, vol. 44(1), pages 83-95, January.
    3. Jonathan Karnon & James Stahl & Alan Brennan & J. Jaime Caro & Javier Mar & Jörgen Möller, 2012. "Modeling Using Discrete Event Simulation," Medical Decision Making, , vol. 32(5), pages 701-711, September.
    4. Anna Heath & Petros Pechlivanoglou, 2022. "Prioritizing Research in an Era of Personalized Medicine: The Potential Value of Unexplained Heterogeneity," Medical Decision Making, , vol. 42(5), pages 649-660, July.
    5. V. J. Roelofs & M. C. Kennedy, 2011. "Sensitivity Analysis and Estimation of Extreme Tail Behavior in Two‐Dimensional Monte Carlo Simulation," Risk Analysis, John Wiley & Sons, vol. 31(10), pages 1597-1609, October.
    6. Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    7. Stijntje W. Dijk & Eline Krijkamp & Natalia Kunst & Jeremy A. Labrecque & Cary P. Gross & Aradhana Pandit & Chia-Ping Lu & Loes E. Visser & John B. Wong & M. G. Myriam Hunink, 2024. "Making Drug Approval Decisions in the Face of Uncertainty: Cumulative Evidence versus Value of Information," Medical Decision Making, , vol. 44(5), pages 512-528, July.
    8. Anna Heath, 2022. "Calculating Expected Value of Sample Information Adjusting for Imperfect Implementation," Medical Decision Making, , vol. 42(5), pages 626-636, July.
    9. Helen A. Dakin & José Leal & Andrew Briggs & Philip Clarke & Rury R. Holman & Alastair Gray, 2020. "Accurately Reflecting Uncertainty When Using Patient-Level Simulation Models to Extrapolate Clinical Trial Data," Medical Decision Making, , vol. 40(4), pages 460-473, May.
    10. Yingying Zhang & Noemi Kreif & Vijay S. GC & Andrea Manca, 2024. "Machine Learning Methods to Estimate Individualized Treatment Effects for Use in Health Technology Assessment," Medical Decision Making, , vol. 44(7), pages 756-769, October.
    11. Yubo Wang & Li-Chia Chen & Kwok-Leung Cheung & Douglas Steinke & Sean P. Gavan, 2025. "Primary endocrine therapy versus surgery plus endocrine therapy for early-stage breast cancer in older women without frailty: a cost-effectiveness and value of implementation analysis," Health Economics Review, Springer, vol. 15(1), pages 1-11, December.
    12. Enoch Yi-Tung Chen & Paul W. Dickman & Mark S. Clements, 2025. "A Multistate Model Incorporating Relative Survival Extrapolation and Mixed Time Scales for Health Technology Assessment," PharmacoEconomics, Springer, vol. 43(3), pages 297-310, March.
    13. Joke Bilcke & Philippe Beutels & Marc Brisson & Mark Jit, 2011. "Accounting for Methodological, Structural, and Parameter Uncertainty in Decision-Analytic Models," Medical Decision Making, , vol. 31(4), pages 675-692, July.
    14. Anna Heath & Gianluca Baio & Ioanna Manolopoulou & Nicky J. Welton, 2024. "Value of Information for Clinical Trial Design: The Importance of Considering All Relevant Comparators," PharmacoEconomics, Springer, vol. 42(5), pages 479-486, May.
    15. Oluwatomi Adetunji & John Bischoff & Christopher J. Willy, 2018. "Managing system obsolescence via multicriteria decision making," Systems Engineering, John Wiley & Sons, vol. 21(4), pages 307-321, July.
    16. Marta Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    17. Jonah Popp & John A. Nyman & Xianghua Luo & Jill Bengtson & Katherine Lust & Lawrence An & Jasjit S. Ahluwalia & Janet L. Thomas, 2018. "Cost-effectiveness of enhancing a Quit-and-Win smoking cessation program for college students," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 19(9), pages 1319-1333, December.
    18. Marta O Soares & L Canto e Castro, 2010. "Simulation or cohort models? Continuous time simulation and discretized Markov models to estimate cost-effectiveness," Working Papers 056cherp, Centre for Health Economics, University of York.
    19. Marta O. Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    20. Manuel Antonio Espinoza & Andrea Manca & Karl Claxton & Mark Sculpher, 2018. "Social value and individual choice: The value of a choice‐based decision‐making process in a collectively funded health system," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 28-40, February.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:45:y:2025:i:2:p:127-142. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.