IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v16y2007i10p1009-1023.html
   My bibliography  Save this article

Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA

Author

Listed:
  • Anthony O'Hagan
  • Matt Stevenson
  • Jason Madan

Abstract

Probabilistic sensitivity analysis (PSA) is required to account for uncertainty in cost‐effectiveness calculations arising from health economic models. The simplest way to perform PSA in practice is by Monte Carlo methods, which involves running the model many times using randomly sampled values of the model inputs. However, this can be impractical when the economic model takes appreciable amounts of time to run. This situation arises, in particular, for patient‐level simulation models (also known as micro‐simulation or individual‐level simulation models), where a single run of the model simulates the health care of many thousands of individual patients. The large number of patients required in each run to achieve accurate estimation of cost‐effectiveness means that only a relatively small number of runs is possible. For this reason, it is often said that PSA is not practical for patient‐level models. We develop a way to reduce the computational burden of Monte Carlo PSA for patient‐level models, based on the algebra of analysis of variance. Methods are presented to estimate the mean and variance of the model output, with formulae for determining optimal sample sizes. The methods are simple to apply and will typically reduce the computational demand very substantially. Copyright © 2006 John Wiley & Sons, Ltd.

Suggested Citation

  • Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023, October.
  • Handle: RePEc:wly:hlthec:v:16:y:2007:i:10:p:1009-1023
    DOI: 10.1002/hec.1199
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.1199
    Download Restriction: no

    File URL: https://libkey.io/10.1002/hec.1199?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. M. D. Stevenson & J. Oakley & J. B. Chilcott, 2004. "Gaussian Process Modeling in Conjunction with Individual Patient Simulation Modeling: A Case Study Describing the Calculation of Cost-Effectiveness Ratios for the Treatment of Established Osteoporosis," Medical Decision Making, , vol. 24(1), pages 89-100, January.
    2. Davies, Ruth & Roderick, Paul & Raftery, James, 2003. "The evaluation of disease prevention and treatment using simulation models," European Journal of Operational Research, Elsevier, vol. 150(1), pages 53-66, October.
    3. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    4. A. David Paltiel & Julie A. Scharfstein & George R. Seage & Elena Losina & Sue J. Goldie & Milton C. Weinstein & Donald E. Craven & Kenneth A. Freedberg, 1998. "A Monte Carlo Simulation of Advanced HIV Disease," Medical Decision Making, , vol. 18(2_suppl), pages 93-105, April.
    5. Alan Brennan & Stephen E. Chick & Ruth Davies, 2006. "A taxonomy of model structures for economic evaluation of health technologies," Health Economics, John Wiley & Sons, Ltd., vol. 15(12), pages 1295-1310, December.
    6. Karl Claxton & Mark Sculpher & Chris McCabe & Andrew Briggs & Ron Akehurst & Martin Buxton & John Brazier & Tony O'Hagan, 2005. "Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra," Health Economics, John Wiley & Sons, Ltd., vol. 14(4), pages 339-347, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Joke Bilcke & Philippe Beutels & Marc Brisson & Mark Jit, 2011. "Accounting for Methodological, Structural, and Parameter Uncertainty in Decision-Analytic Models," Medical Decision Making, , vol. 31(4), pages 675-692, July.
    2. Marta O. Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    3. Philippe Laramée & Aurélie Millier & Nora Rahhali & Olivier Cristeau & Samuel Aballéa & Clément François & Ylana Chalem & Mondher Toumi & Jürgen Rehm, 2016. "A Trial-Based Predictive Microsimulation Assessing the Public Health Benefits of Nalmefene and Psychosocial Support for the Reduction of Alcohol Consumption in Alcohol Dependence," Applied Health Economics and Health Policy, Springer, vol. 14(4), pages 493-505, August.
    4. Jonah Popp & John A. Nyman & Xianghua Luo & Jill Bengtson & Katherine Lust & Lawrence An & Jasjit S. Ahluwalia & Janet L. Thomas, 2018. "Cost-effectiveness of enhancing a Quit-and-Win smoking cessation program for college students," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 19(9), pages 1319-1333, December.
    5. Oluwatomi Adetunji & John Bischoff & Christopher J. Willy, 2018. "Managing system obsolescence via multicriteria decision making," Systems Engineering, John Wiley & Sons, vol. 21(4), pages 307-321, July.
    6. Marta Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    7. Jonathan Karnon & James Stahl & Alan Brennan & J. Jaime Caro & Javier Mar & Jörgen Möller, 2012. "Modeling Using Discrete Event Simulation," Medical Decision Making, , vol. 32(5), pages 701-711, September.
    8. Marta O Soares & L Canto e Castro, 2010. "Simulation or cohort models? Continuous time simulation and discretized Markov models to estimate cost-effectiveness," Working Papers 056cherp, Centre for Health Economics, University of York.
    9. Helen A. Dakin & José Leal & Andrew Briggs & Philip Clarke & Rury R. Holman & Alastair Gray, 2020. "Accurately Reflecting Uncertainty When Using Patient-Level Simulation Models to Extrapolate Clinical Trial Data," Medical Decision Making, , vol. 40(4), pages 460-473, May.
    10. V. J. Roelofs & M. C. Kennedy, 2011. "Sensitivity Analysis and Estimation of Extreme Tail Behavior in Two‐Dimensional Monte Carlo Simulation," Risk Analysis, John Wiley & Sons, vol. 31(10), pages 1597-1609, October.
    11. Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    12. Jagpreet Chhatwal & Tianhua He, 2015. "Economic Evaluations with Agent-Based Modelling: An Introduction," PharmacoEconomics, Springer, vol. 33(5), pages 423-433, May.
    13. Salah Ghabri & Françoise F. Hamers & Jean Michel Josselin, 2016. "Exploring Uncertainty in Economic Evaluations of Drugs and Medical Devices: Lessons from the First Review of Manufacturers’ Submissions to the French National Authority for Health," PharmacoEconomics, Springer, vol. 34(6), pages 617-624, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023.
    2. Marta Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    3. Marta O. Soares & Luísa Canto e Castro, 2012. "Continuous Time Simulation and Discretized Models for Cost-Effectiveness Analysis," PharmacoEconomics, Springer, vol. 30(12), pages 1101-1117, December.
    4. Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    5. Marta O Soares & L Canto e Castro, 2010. "Simulation or cohort models? Continuous time simulation and discretized Markov models to estimate cost-effectiveness," Working Papers 056cherp, Centre for Health Economics, University of York.
    6. Isaac Corro Ramos & Maureen P. M. H. Rutten-van Mölken & Maiwenn J. Al, 2013. "The Role of Value-of-Information Analysis in a Health Care Research Priority Setting," Medical Decision Making, , vol. 33(4), pages 472-489, May.
    7. Olivier Ethgen & Baudouin Standaert, 2012. "Population–versus Cohort–Based Modelling Approaches," PharmacoEconomics, Springer, vol. 30(3), pages 171-181, March.
    8. Alan Brennan & Stephen E. Chick & Ruth Davies, 2006. "A taxonomy of model structures for economic evaluation of health technologies," Health Economics, John Wiley & Sons, Ltd., vol. 15(12), pages 1295-1310, December.
    9. Tiago M. de Carvalho & Eveline A. M. Heijnsdijk & Luc Coffeng & Harry J. de Koning, 2019. "Evaluating Parameter Uncertainty in a Simulation Model of Cancer Using Emulators," Medical Decision Making, , vol. 39(4), pages 405-413, May.
    10. Belfiore, Alessandra & Scaletti, Alessandro & Lavorato, Domenica & Cuccurullo, Corrado, 2023. "The long process by which HTA became a paradigm: A longitudinal conceptual structure analysis," Health Policy, Elsevier, vol. 127(C), pages 74-79.
    11. Christopher H. Jackson & Laura Bojke & Simon G. Thompson & Karl Claxton & Linda D. Sharples, 2011. "A Framework for Addressing Structural Uncertainty in Decision Models," Medical Decision Making, , vol. 31(4), pages 662-674, July.
    12. Cannon, Jeffrey W. & Mueller, Ute A. & Hornbuckle, Janet & Larson, Ann & Simmer, Karen & Newnham, John P. & Doherty, Dorota A., 2013. "Economic implications of poor access to antenatal care in rural and remote Western Australian Aboriginal communities: An individual sampling model of pregnancy," European Journal of Operational Research, Elsevier, vol. 226(2), pages 313-324.
    13. M D Stevenson & J E Oakley & S E Chick & K Chalkidou, 2009. "The cost-effectiveness of surgical instrument management policies to reduce the risk of vCJD transmission to humans," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 60(4), pages 506-518, April.
    14. Ji-Hee Youn & Matt D. Stevenson & Praveen Thokala & Katherine Payne & Maria Goddard, 2019. "Modeling the Economic Impact of Interventions for Older Populations with Multimorbidity: A Method of Linking Multiple Single-Disease Models," Medical Decision Making, , vol. 39(7), pages 842-856, October.
    15. A. E. Ades & Karl Claxton & Mark Sculpher, 2006. "Evidence synthesis, parameter correlation and probabilistic sensitivity analysis," Health Economics, John Wiley & Sons, Ltd., vol. 15(4), pages 373-381, April.
    16. S. Cucurachi & E. Borgonovo & R. Heijungs, 2016. "A Protocol for the Global Sensitivity Analysis of Impact Assessment Models in Life Cycle Assessment," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 357-377, February.
    17. Heß, Michael (Ed.) & Schlieter, Hannes (Ed.), 2014. "Modellierung im Gesundheitswesen: Tagungsband des Workshops im Rahmen der Modellierung 2014," ICB Research Reports 57, University Duisburg-Essen, Institute for Computer Science and Business Information Systems (ICB).
    18. Jakub Bijak & Jason D. Hilton & Eric Silverman & Viet Dung Cao, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.
    19. Acharki, Naoufal & Bertoncello, Antoine & Garnier, Josselin, 2023. "Robust prediction interval estimation for Gaussian processes by cross-validation method," Computational Statistics & Data Analysis, Elsevier, vol. 178(C).
    20. Aditya Sai & Carolina Vivas-Valencia & Thomas F. Imperiale & Nan Kong, 2019. "Multiobjective Calibration of Disease Simulation Models Using Gaussian Processes," Medical Decision Making, , vol. 39(5), pages 540-552, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:16:y:2007:i:10:p:1009-1023. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.