IDEAS home Printed from https://ideas.repec.org/a/sae/medema/v45y2025i8p976-986.html

Meta-Modeling as a Variance-Reduction Technique for Stochastic Model–Based Cost-Effectiveness Analyses

Author

Listed:
  • Zongbo Li

    (Division of Health Policy and Management, University of Minnesota, School of Public Health, Minneapolis, MN, USA)

  • Gregory S. Knowlton

    (Division of Health Policy and Management, University of Minnesota, School of Public Health, Minneapolis, MN, USA)

  • Margo M. Wheatley

    (Division of Health Policy and Management, University of Minnesota, School of Public Health, Minneapolis, MN, USA)

  • Samuel M. Jenness

    (Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, GA, USA)

  • Eva A. Enns

    (Division of Health Policy and Management, University of Minnesota, School of Public Health, Minneapolis, MN, USA)

Abstract

Purpose When using stochastic models for cost-effectiveness analysis (CEA), run-to-run outcome variability arising from model stochasticity can sometimes exceed the change in outcomes resulting from an intervention, especially when individual-level efficacy is small, leading to counterintuitive results. This issue is compounded for probabilistic sensitivity analyses (PSAs), in which stochastic noise can obscure the influence of parameter uncertainty. This study evaluates meta-modeling as a variance-reduction technique to mitigate stochastic noise while preserving parameter uncertainty in PSAs. Methods We applied meta-modeling to 2 simulation models: 1) a 4-state Sick-Sicker model and 2) an agent-based HIV transmission model among men who have sex with men (MSM). We conducted a PSA and applied 3 meta-modeling techniques—linear regression, generalized additive models, and artificial neural networks—to reduce stochastic noise. Model performance was assessed using R 2 and root mean squared error (RMSE) values on a validation dataset. We compared PSA results by examining scatter plots of incremental costs and quality-adjusted life-years (QALYs), cost-effectiveness acceptability curves (CEACs), and the occurrence of unintuitive results, such as interventions appearing to reduce QALYs due to stochastic noise. Results In the Sick-Sicker model, stochastic noise increased variance in incremental costs and QALYs. Applying meta-modeling techniques substantially reduced this variance and nearly eliminated unintuitive results, with R 2 and RMSE values indicating good model fit. In the HIV agent-based model, all 3 meta-models effectively reduced outcome variability while retaining parameter uncertainty, yielding more informative CEACs with higher probabilities of being cost-effective for the optimal strategy. Conclusions Meta-modeling effectively reduces stochastic noise in simulation models while maintaining parameter uncertainty in PSA, enhancing the reliability of CEA results without requiring an impractical number of simulations. Highlights When using complex stochastic models for cost-effectiveness analysis (CEA), stochastic noise can overwhelm intervention effects and obscure the impact of parameter uncertainty on CEA outcomes in probabilistic sensitivity analysis (PSA). Meta-modeling offers a solution by effectively reducing stochastic noise in complex stochastic simulation models without increasing computational burden, thereby improving the interpretability of PSA results.

Suggested Citation

  • Zongbo Li & Gregory S. Knowlton & Margo M. Wheatley & Samuel M. Jenness & Eva A. Enns, 2025. "Meta-Modeling as a Variance-Reduction Technique for Stochastic Model–Based Cost-Effectiveness Analyses," Medical Decision Making, , vol. 45(8), pages 976-986, November.
  • Handle: RePEc:sae:medema:v:45:y:2025:i:8:p:976-986
    DOI: 10.1177/0272989X251352210
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0272989X251352210
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0272989X251352210?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Natasha Stout & Sue Goldie, 2008. "Keeping the noise down: common random numbers for disease simulation modeling," Health Care Management Science, Springer, vol. 11(4), pages 399-406, December.
    2. Bas Groot Koerkamp & Milton C. Weinstein & Theo Stijnen & M.H. Heijenbrok-Kal & M.G. Myriam Hunink, 2010. "Uncertainty and Patient Heterogeneity in Medical Decision Models," Medical Decision Making, , vol. 30(2), pages 194-205, March.
    3. Steven M. Shechter & Andrew J. Schaefer & R. Scott Braithwaite & Mark S. Roberts, 2006. "Increasing the Efficiency of Monte Carlo Cohort Simulations with Variance Reduction Techniques," Medical Decision Making, , vol. 26(5), pages 550-553, September.
    4. Eline M. Krijkamp & Fernando Alarid-Escudero & Eva A. Enns & Hawre J. Jalal & M. G. Myriam Hunink & Petros Pechlivanoglou, 2018. "Microsimulation Modeling for Health Decision Sciences Using R: A Tutorial," Medical Decision Making, , vol. 38(3), pages 400-422, April.
    5. Huaiyang Zhong & Margaret L. Brandeau & Golnaz Eftekhari Yazdi & Jianing Wang & Shayla Nolen & Liesl Hagan & William W. Thompson & Sabrina A. Assoumou & Benjamin P. Linas & Joshua A. Salomon, 2022. "Metamodeling for Policy Simulations with Multivariate Outcomes," Medical Decision Making, , vol. 42(7), pages 872-884, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ziyi Lin & Andrew Briggs, 2026. "Beyond the States: Developing a Discrete Event Simulation Model Using R," PharmacoEconomics, Springer, vol. 44(4), pages 389-408, April.
    2. Maarten Ijzerman & Lotte Steuten, 2011. "Early assessment of medical technologies to inform product development and market access," Applied Health Economics and Health Policy, Springer, vol. 9(5), pages 331-347, September.
    3. Fernando Alarid-Escudero & Eline Krijkamp & Eva A. Enns & Alan Yang & M. G. Myriam Hunink & Petros Pechlivanoglou & Hawre Jalal, 2023. "A Tutorial on Time-Dependent Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example," Medical Decision Making, , vol. 43(1), pages 21-41, January.
    4. Jeremy D. Goldhaber-Fiebert & Hawre Jalal & Fernando Alarid-Escudero, 2025. "Microsimulation Estimates of Decision Uncertainty and Value of Information Are Biased but Consistent," Medical Decision Making, , vol. 45(2), pages 127-142, February.
    5. Mei-Chin Su & Yi-Jen Wang & Tzeng-Ji Chen & Shiao-Hui Chiu & Hsiao-Ting Chang & Mei-Shu Huang & Li-Hui Hu & Chu-Chuan Li & Su-Ju Yang & Jau-Ching Wu & Yu-Chun Chen, 2020. "Assess the Performance and Cost-Effectiveness of LACE and HOSPITAL Re-Admission Prediction Models as a Risk Management Tool for Home Care Patients: An Evaluation Study of a Medical Center Affiliated Home Care Unit in Taiwan," IJERPH, MDPI, vol. 17(3), pages 1-17, February.
    6. Anna Heath & Petros Pechlivanoglou, 2022. "Prioritizing Research in an Era of Personalized Medicine: The Potential Value of Unexplained Heterogeneity," Medical Decision Making, , vol. 42(5), pages 649-660, July.
    7. González-Parra, Gilberto & Villanueva-Oller, Javier & Navarro-González, F.J. & Ceberio, Josu & Luebben, Giulia, 2024. "A network-based model to assess vaccination strategies for the COVID-19 pandemic by using Bayesian optimization," Chaos, Solitons & Fractals, Elsevier, vol. 181(C).
    8. Isaac Corro Ramos & Talitha Feenstra & Salah Ghabri & Maiwenn Al, 2024. "Evaluating the Validation Process: Embracing Complexity and Transparency in Health Economic Modelling," PharmacoEconomics, Springer, vol. 42(7), pages 715-719, July.
    9. Yi-Shu Lin & James F O’Mahony & Joost Rosmalen, 2023. "A Simple Cost-Effectiveness Model of Screening: An Open-Source Teaching and Research Tool Coded in R," PharmacoEconomics - Open, Springer, vol. 7(4), pages 507-523, July.
    10. Peter Shewmaker & Stavroula A. Chrysanthopoulou & Rowan Iskandar & Derek Lake & Earic Jutkowitz, 2022. "Microsimulation Model Calibration with Approximate Bayesian Computation in R: A Tutorial," Medical Decision Making, , vol. 42(5), pages 557-570, July.
    11. Koen Degeling & Jonathan Karnon & Michiel Ven & Alan Brennan & Hendrik Koffijberg, 2025. "Discrete Event Simulation in R using the ‘Simmer’ Package for Health Economic Modelling: A Tutorial and Illustration in Colon Cancer," Applied Health Economics and Health Policy, Springer, vol. 23(6), pages 961-975, November.
    12. Alexandra Moskalewicz & Sumit Gupta & Paul C. Nathan & Petros Pechlivanoglou, 2025. "Development of a Microsimulation Model to Project the Future Prevalence of Childhood Cancer in Ontario, Canada," Medical Decision Making, , vol. 45(3), pages 245-256, April.
    13. Troost, Christian & Huber, Robert & Bell, Andrew R. & van Delden, Hedwig & Filatova, Tatiana & Le, Quang Bao & Lippe, Melvin & Niamir, Leila & Polhill, J. Gareth & Sun, Zhanli & Berger, Thomas, 2023. "How to keep it adequate: A protocol for ensuring validity in agent-based simulation," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 159, pages 1-21.
    14. repec:ijm:journl:v109:y:2017:i:1:p:167-200 is not listed on IDEAS
    15. James F Burke & Kenneth M Langa & Rodney A Hayward & Roger L Albin, 2014. "Modeling Test and Treatment Strategies for Presymptomatic Alzheimer Disease," PLOS ONE, Public Library of Science, vol. 9(12), pages 1-20, December.
    16. Ellen X. Y. Hu & Evelien S. van Hoorn & Isabel R. A. Retel Helmrich & Susanne Muehlschlegel & Judith A. C. Rietjens & Hester F. Lingsma, 2025. "Facilitators and Barriers of the Use of Prognostic Models for Clinical Decision Making in Acute Neurologic Care: A Systematic Review," Medical Decision Making, , vol. 45(6), pages 753-770, August.
    17. Matthijs M. Versteegh & Simone A. Huygens & Inigo Bermejo & Sabine Grimm & Marieke Pierik & Tessa Römkens & Fiona Schaik & Peter Wahab, 2025. "Cost–Utility Analysis of Treatment Sequences for Moderate-to-Severe Crohn’s Disease," PharmacoEconomics, Springer, vol. 43(12), pages 1405-1417, December.
    18. Kokina, Julia & Blanchette, Shay & Davenport, Thomas H. & Pachamanova, Dessislava, 2025. "Challenges and opportunities for artificial intelligence in auditing: Evidence from the field," International Journal of Accounting Information Systems, Elsevier, vol. 56(C).
    19. Stavroula A. Chrysanthopoulou & Carolyn M. Rutter & Constantine A. Gatsonis, 2021. "Bayesian versus Empirical Calibration of Microsimulation Models: A Comparative Analysis," Medical Decision Making, , vol. 41(6), pages 714-726, August.
    20. Yingying Zhang & Noemi Kreif & Vijay S. GC & Andrea Manca, 2024. "Machine Learning Methods to Estimate Individualized Treatment Effects for Use in Health Technology Assessment," Medical Decision Making, , vol. 44(7), pages 756-769, October.
    21. Hendriek Boshuizen & Stefan Lhachimi & Pieter Baal & Rudolf Hoogenveen & Henriette Smit & Johan Mackenbach & Wilma Nusselder, 2012. "The DYNAMO-HIA Model: An Efficient Implementation of a Risk Factor/Chronic Disease Markov Model for Use in Health Impact Assessment (HIA)," Demography, Springer;Population Association of America (PAA), vol. 49(4), pages 1259-1283, November.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:medema:v:45:y:2025:i:8:p:976-986. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.