IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v31y2011i10p1597-1609.html
   My bibliography  Save this article

Sensitivity Analysis and Estimation of Extreme Tail Behavior in Two‐Dimensional Monte Carlo Simulation

Author

Listed:
  • V. J. Roelofs
  • M. C. Kennedy

Abstract

Two‐dimensional Monte Carlo simulation is frequently used to implement probabilistic risk models, as it allows for uncertainty and variability to be quantified separately. In many cases, we are interested in the proportion of individuals from a variable population exceeding a critical threshold, together with uncertainty about this proportion. In this article we introduce a new method that can accurately estimate these quantities much more efficiently than conventional algorithms. We also show how those model parameters having the greatest impact on the probabilities of rare events can be quickly identified via this method. The algorithm combines elements from well‐established statistical techniques in extreme value theory and Bayesian analysis of computer models. We demonstrate the practical application of these methods with a simple example, in which the true distributions are known exactly, and also with a more realistic model of microbial contamination of milk with seven parameters. For the latter, sensitivity analysis (SA) is shown to identify the two inputs explaining the majority of variation in distribution tail behavior. In the subsequent prediction of probabilities of large contamination events, similar results are obtained using the new approach taking 43 seconds or the conventional simulation that requires more than 3 days.

Suggested Citation

  • V. J. Roelofs & M. C. Kennedy, 2011. "Sensitivity Analysis and Estimation of Extreme Tail Behavior in Two‐Dimensional Monte Carlo Simulation," Risk Analysis, John Wiley & Sons, vol. 31(10), pages 1597-1609, October.
  • Handle: RePEc:wly:riskan:v:31:y:2011:i:10:p:1597-1609
    DOI: 10.1111/j.1539-6924.2011.01599.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.2011.01599.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1539-6924.2011.01599.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023, October.
    2. Anthony O'Hagan & Matt Stevenson & Jason Madan, 2007. "Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA," Health Economics, John Wiley & Sons, Ltd., vol. 16(10), pages 1009-1023.
    3. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    4. Jeremy Oakley, 2002. "Bayesian inference for the uncertainty distribution of computer model outputs," Biometrika, Biometrika Trust, vol. 89(4), pages 769-784, December.
    5. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    6. Nicolas Miconnet & Marie Cornu & Annie Beaufort & Laurent Rosso & Jean‐Baptiste Denis, 2005. "Uncertainty Distribution Associated with Estimating a Proportion in Microbial Risk Assessment," Risk Analysis, John Wiley & Sons, vol. 25(1), pages 39-48, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wang, Zhonglai & Liu, Jing & Yu, Shui, 2020. "Time-variant reliability prediction for dynamic systems using partial information," Reliability Engineering and System Safety, Elsevier, vol. 195(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jakub Bijak & Jason D. Hilton & Eric Silverman & Viet Dung Cao, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.
    2. Petropoulos, G. & Wooster, M.J. & Carlson, T.N. & Kennedy, M.C. & Scholze, M., 2009. "A global Bayesian sensitivity analysis of the 1d SimSphere soil–vegetation–atmospheric transfer (SVAT) model using Gaussian model emulation," Ecological Modelling, Elsevier, vol. 220(19), pages 2427-2440.
    3. Oakley, Jeremy E. & Brennan, Alan & Tappenden, Paul & Chilcott, Jim, 2010. "Simulation sample sizes for Monte Carlo partial EVPI calculations," Journal of Health Economics, Elsevier, vol. 29(3), pages 468-477, May.
    4. Priscilla Avegliano & Jaime Simão Sichman, 2023. "Equation-Based Versus Agent-Based Models: Why Not Embrace Both for an Efficient Parameter Calibration?," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 26(4), pages 1-3.
    5. Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
    6. O’Hagan, A., 2006. "Bayesian analysis of computer code outputs: A tutorial," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1290-1300.
    7. Jonathan Karnon & James Stahl & Alan Brennan & J. Jaime Caro & Javier Mar & Jörgen Möller, 2012. "Modeling Using Discrete Event Simulation," Medical Decision Making, , vol. 32(5), pages 701-711, September.
    8. Ioannis Andrianakis & Ian R Vernon & Nicky McCreesh & Trevelyan J McKinley & Jeremy E Oakley & Rebecca N Nsubuga & Michael Goldstein & Richard G White, 2015. "Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda," PLOS Computational Biology, Public Library of Science, vol. 11(1), pages 1-18, January.
    9. Marc Kennedy & Clive Anderson & Anthony O'Hagan & Mark Lomas & Ian Woodward & John Paul Gosling & Andreas Heinemeyer, 2008. "Quantifying uncertainty in the biospheric carbon flux for England and Wales," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 171(1), pages 109-135, January.
    10. Kennedy, Marc C. & Anderson, Clive W. & Conti, Stefano & O’Hagan, Anthony, 2006. "Case studies in Gaussian process modelling of computer codes," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1301-1309.
    11. Antony M. Overstall & David C. Woods, 2016. "Multivariate emulation of computer simulators: model selection and diagnostics with application to a humanitarian relief model," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 65(4), pages 483-505, August.
    12. Andrianakis, Ioannis & Challenor, Peter G., 2012. "The effect of the nugget on Gaussian process emulators of computer models," Computational Statistics & Data Analysis, Elsevier, vol. 56(12), pages 4215-4228.
    13. Raymond K. W. Wong & Curtis B. Storlie & Thomas C. M. Lee, 2017. "A frequentist approach to computer model calibration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(2), pages 635-648, March.
    14. Reichert, P. & White, G. & Bayarri, M.J. & Pitman, E.B., 2011. "Mechanism-based emulation of dynamic simulation models: Concept and application in hydrology," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1638-1655, April.
    15. SungKu Kang & Ran Jin & Xinwei Deng & Ron S. Kenett, 2023. "Challenges of modeling and analysis in cybermanufacturing: a review from a machine learning and computation perspective," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 415-428, February.
    16. Manfren, Massimiliano & Aste, Niccolò & Moshksar, Reza, 2013. "Calibration and uncertainty analysis for computer models – A meta-model based approach for integrated building energy simulation," Applied Energy, Elsevier, vol. 103(C), pages 627-641.
    17. Geoffrey Fairchild & Kyle S. Hickmann & Susan M. Mniszewski & Sara Y. Del Valle & James M. Hyman, 2014. "Optimizing human activity patterns using global sensitivity analysis," Computational and Mathematical Organization Theory, Springer, vol. 20(4), pages 394-416, December.
    18. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    19. Zitrou, A. & Bedford, T. & Daneshkhah, A., 2013. "Robustness of maintenance decisions: Uncertainty modelling and value of information," Reliability Engineering and System Safety, Elsevier, vol. 120(C), pages 60-71.
    20. Helen A. Dakin & José Leal & Andrew Briggs & Philip Clarke & Rury R. Holman & Alastair Gray, 2020. "Accurately Reflecting Uncertainty When Using Patient-Level Simulation Models to Extrapolate Clinical Trial Data," Medical Decision Making, , vol. 40(4), pages 460-473, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:31:y:2011:i:10:p:1597-1609. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.