IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v113y2013icp30-41.html
   My bibliography  Save this article

Analysis of computationally demanding models with continuous and categorical inputs

Author

Listed:
  • Storlie, Curtis B.
  • Reich, Brian J.
  • Helton, Jon C.
  • Swiler, Laura P.
  • Sallaberry, Cedric J.

Abstract

The analysis of many physical and engineering problems involves running complex computational models (e.g., simulation models and computer codes). With problems of this type, it is important to understand the relationships between the input (whose values are often imprecisely known) and the output variables, and to characterize the uncertainty in the output. Often, some of the input variables are categorical in nature (e.g., pointer variables to alternative models or different types of material, etc.). A computational model that sufficiently represents reality is often very costly in terms of run time. When the models are computationally demanding, meta-model approaches to their analysis have been shown to be very useful. However, the most popular meta-models for computational computer models do not explicitly allow for categorical input variables. In this case, categorical inputs are simply ordered in some way and treated as continuous variables in the estimation of a meta-model. In many cases, this can lead to undesirable and misleading results. In this paper, two meta-models based on functional ANOVA decomposition are presented that explicitly allow for an appropriate treatment of categorical inputs. The effectiveness of the presented meta-models in the analysis of models with continuous and categorical inputs is illustrated with several test cases and also with results from a real analysis.

Suggested Citation

  • Storlie, Curtis B. & Reich, Brian J. & Helton, Jon C. & Swiler, Laura P. & Sallaberry, Cedric J., 2013. "Analysis of computationally demanding models with continuous and categorical inputs," Reliability Engineering and System Safety, Elsevier, vol. 113(C), pages 30-41.
  • Handle: RePEc:eee:reensy:v:113:y:2013:i:c:p:30-41
    DOI: 10.1016/j.ress.2012.11.018
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832012002487
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ress.2012.11.018?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Storlie, Curtis B. & Helton, Jon C., 2008. "Multiple predictor smoothing methods for sensitivity analysis: Description of techniques," Reliability Engineering and System Safety, Elsevier, vol. 93(1), pages 28-54.
    2. Gramacy, Robert B & Lee, Herbert K. H, 2008. "Bayesian Treed Gaussian Process Models With an Application to Computer Modeling," Journal of the American Statistical Association, American Statistical Association, vol. 103(483), pages 1119-1130.
    3. Montserrat Fuentes, 2002. "Spectral methods for nonstationary spatial processes," Biometrika, Biometrika Trust, vol. 89(1), pages 197-210, March.
    4. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    5. Storlie, Curtis B. & Helton, Jon C., 2008. "Multiple predictor smoothing methods for sensitivity analysis: Example results," Reliability Engineering and System Safety, Elsevier, vol. 93(1), pages 55-77.
    6. Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    7. Marrel, Amandine & Iooss, Bertrand & Van Dorpe, François & Volkova, Elena, 2008. "An efficient methodology for modeling complex computer codes with Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4731-4744, June.
    8. Jeremy E. Oakley & Anthony O'Hagan, 2004. "Probabilistic sensitivity analysis of complex models: a Bayesian approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(3), pages 751-769, August.
    9. Helton, J.C. & Johnson, J.D. & Sallaberry, C.J. & Storlie, C.B., 2006. "Survey of sampling-based methods for uncertainty and sensitivity analysis," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1175-1209.
    10. Jeremy Oakley, 2002. "Bayesian inference for the uncertainty distribution of computer model outputs," Biometrika, Biometrika Trust, vol. 89(4), pages 769-784, December.
    11. Ming Yuan & Yi Lin, 2006. "Model selection and estimation in regression with grouped variables," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(1), pages 49-67, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    2. Di Maio, Francesco & Bandini, Alessandro & Zio, Enrico & Alberola, Sofia Carlos & Sanchez-Saez, Francisco & Martorell, Sebastián, 2016. "Bootstrapped-ensemble-based Sensitivity Analysis of a trace thermal-hydraulic model based on a limited number of PWR large break loca simulations," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 122-134.
    3. K. Sham Bhat & David S. Mebane & Priyadarshi Mahapatra & Curtis B. Storlie, 2017. "Upscaling Uncertainty with Dynamic Discrepancy for a Multi-Scale Carbon Capture System," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(520), pages 1453-1467, October.
    4. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    5. Zhai, Qingqing & Yang, Jun & Xie, Min & Zhao, Yu, 2014. "Generalized moment-independent importance measures based on Minkowski distance," European Journal of Operational Research, Elsevier, vol. 239(2), pages 449-455.
    6. Emanuele Borgonovo & Gordon B. Hazen & Elmar Plischke, 2016. "A Common Rationale for Global Sensitivity Measures and Their Estimation," Risk Analysis, John Wiley & Sons, vol. 36(10), pages 1871-1895, October.
    7. Curtis B. Storlie & William A. Lane & Emily M. Ryan & James R. Gattiker & David M. Higdon, 2015. "Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 68-82, March.
    8. Raymond K. W. Wong & Curtis B. Storlie & Thomas C. M. Lee, 2017. "A frequentist approach to computer model calibration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(2), pages 635-648, March.
    9. J. Rohmer & S. Lecacheux & R. Pedreros & H. Quetelard & F. Bonnardot & D. Idier, 2016. "Dynamic parameter sensitivity in numerical modelling of cyclone-induced waves: a multi-look approach using advanced meta-modelling techniques," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 84(3), pages 1765-1792, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Storlie, Curtis B. & Swiler, Laura P. & Helton, Jon C. & Sallaberry, Cedric J., 2009. "Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models," Reliability Engineering and System Safety, Elsevier, vol. 94(11), pages 1735-1763.
    2. Touzani, Samir & Busby, Daniel, 2013. "Smoothing spline analysis of variance approach for global sensitivity analysis of computer codes," Reliability Engineering and System Safety, Elsevier, vol. 112(C), pages 67-81.
    3. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    4. Blatman, Géraud & Sudret, Bruno, 2010. "Efficient computation of global sensitivity indices using sparse polynomial chaos expansions," Reliability Engineering and System Safety, Elsevier, vol. 95(11), pages 1216-1229.
    5. Helton, Jon C. & Johnson, Jay D. & Sallaberry, Cédric J., 2011. "Quantification of margins and uncertainties: Example analyses from reactor safety and radioactive waste disposal involving the separation of aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1014-1033.
    6. Hansen, C.W. & Behie, G.A. & Bier, A. & Brooks, K.M. & Chen, Y. & Helton, J.C. & Hommel, S.P. & Lee, K.P. & Lester, B. & Mattie, P.D. & Mehta, S. & Miller, S.P. & Sallaberry, C.J. & Sevougian, S.D. & , 2014. "Uncertainty and sensitivity analysis for the nominal scenario class in the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 122(C), pages 272-296.
    7. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Margins associated with loss of assured safety for systems with multiple weak links and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    8. Helton, Jon C. & Hansen, Clifford W. & Sallaberry, Cédric J., 2012. "Uncertainty and sensitivity analysis in performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 44-63.
    9. Di Maio, Francesco & Bandini, Alessandro & Zio, Enrico & Alberola, Sofia Carlos & Sanchez-Saez, Francisco & Martorell, Sebastián, 2016. "Bootstrapped-ensemble-based Sensitivity Analysis of a trace thermal-hydraulic model based on a limited number of PWR large break loca simulations," Reliability Engineering and System Safety, Elsevier, vol. 153(C), pages 122-134.
    10. Daniel W. Gladish & Daniel E. Pagendam & Luk J. M. Peeters & Petra M. Kuhnert & Jai Vaze, 2018. "Emulation Engines: Choice and Quantification of Uncertainty for Complex Hydrological Models," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 23(1), pages 39-62, March.
    11. Helton, Jon C. & Pilch, Martin & Sallaberry, Cédric J., 2014. "Probability of loss of assured safety in systems with multiple time-dependent failure modes: Representations with aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 124(C), pages 171-200.
    12. Helton, Jon C., 2011. "Quantification of margins and uncertainties: Conceptual and computational basis," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 976-1013.
    13. Tatsuya Sakurahara & Seyed Reihani & Ernie Kee & Zahra Mohaghegh, 2020. "Global importance measure methodology for integrated probabilistic risk assessment," Journal of Risk and Reliability, , vol. 234(2), pages 377-396, April.
    14. Curtis B. Storlie & William A. Lane & Emily M. Ryan & James R. Gattiker & David M. Higdon, 2015. "Calibration of Computational Models With Categorical Parameters and Correlated Outputs via Bayesian Smoothing Spline ANOVA," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 68-82, March.
    15. Jakub Bijak & Jason D. Hilton & Eric Silverman & Viet Dung Cao, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.
    16. Petropoulos, G. & Wooster, M.J. & Carlson, T.N. & Kennedy, M.C. & Scholze, M., 2009. "A global Bayesian sensitivity analysis of the 1d SimSphere soil–vegetation–atmospheric transfer (SVAT) model using Gaussian model emulation," Ecological Modelling, Elsevier, vol. 220(19), pages 2427-2440.
    17. Martorell, S. & Villamizar, M. & Martón, I. & Villanueva, J.F. & Carlos, S. & Sánchez, A.I., 2014. "Evaluation of risk impact of changes to surveillance requirements addressing model and parameter uncertainties," Reliability Engineering and System Safety, Elsevier, vol. 126(C), pages 153-165.
    18. Tang, Zhang-Chun & Zuo, Ming J. & Xiao, Ningcong, 2016. "An efficient method for evaluating the effect of input parameters on the integrity of safety systems," Reliability Engineering and System Safety, Elsevier, vol. 145(C), pages 111-123.
    19. V. J. Roelofs & M. C. Kennedy, 2011. "Sensitivity Analysis and Estimation of Extreme Tail Behavior in Two‐Dimensional Monte Carlo Simulation," Risk Analysis, John Wiley & Sons, vol. 31(10), pages 1597-1609, October.
    20. Mara, Thierry A. & Tarantola, Stefano, 2012. "Variance-based sensitivity indices for models with dependent inputs," Reliability Engineering and System Safety, Elsevier, vol. 107(C), pages 115-121.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:113:y:2013:i:c:p:30-41. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.