IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v111y2013icp273-286.html
   My bibliography  Save this article

A sequential approach for stochastic computer model calibration and prediction

Author

Listed:
  • Yuan, Jun
  • Ng, Szu Hui

Abstract

Computer models are widely used to simulate complex and costly real processes and systems. When the computer model is used to assess and certify the real system for decision making, it is often important to calibrate the computer model so as to improve the model’s predictive accuracy. A sequential approach is proposed in this paper for stochastic computer model calibration and prediction. More precisely, we propose a surrogate based Bayesian approach for stochastic computer model calibration which accounts for various uncertainties including the calibration parameter uncertainty in the follow up prediction and computer model analysis. We derive the posterior distribution of the calibration parameter and the predictive distributions for both the real process and the computer model which quantify the calibration and prediction uncertainty and provide the analytical calibration and prediction results. We also derive the predictive distribution of the discrepancy term between the real process and the computer model that can be used to validate the computer model. Furthermore, in order to efficiently use limited data resources to obtain a better calibration and prediction performance, we propose a two-stage sequential approach which can effectively allocate the limited resources. The accuracy and efficiency of the proposed approach are illustrated by the numerical examples.

Suggested Citation

  • Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.
    • Handle: RePEc:eee:reensy:v:111:y:2013:i:c:p:273-286
    DOI: 10.1016/j.ress.2012.11.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832012002335
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2012.11.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bruce Ankenman & Barry L. Nelson & Jeremy Staum, 2010. "Stochastic Kriging for Simulation Metamodeling," Operations Research, INFORMS, vol. 58(2), pages 371-382, April.
    2. Kanso, A. & Chebbo, G. & Tassin, B., 2006. "Application of MCMC–GSA model calibration method to urban runoff quality modeling," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1398-1405.
    3. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    4. Henderson, Daniel A. & Boys, Richard J. & Krishnan, Kim J. & Lawless, Conor & Wilkinson, Darren J., 2009. "Bayesian Emulation and Calibration of a Stochastic Computer Model of Mitochondrial DNA Deletions in Substantia Nigra Neurons," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 76-87.
    5. Campbell, Katherine, 2006. "Statistical calibration of computer simulations," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1358-1363.
    6. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    7. Williams, Brian J. & Loeppky, Jason L. & Moore, Leslie M. & Macklem, Mason S., 2011. "Batch sequential design to achieve predictive maturity with calibrated computer models," Reliability Engineering and System Safety, Elsevier, vol. 96(9), pages 1208-1219.
    8. Trucano, T.G. & Swiler, L.P. & Igusa, T. & Oberkampf, W.L. & Pilch, M., 2006. "Calibration, validation, and sensitivity analysis: What's what," Reliability Engineering and System Safety, Elsevier, vol. 91(10), pages 1331-1357.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Park, Inseok & Grandhi, Ramana V., 2014. "A Bayesian statistical method for quantifying model form uncertainty and two model combination methods," Reliability Engineering and System Safety, Elsevier, vol. 129(C), pages 46-56.
    2. Yuan, Jun & Nian, Victor & Su, Bin, 2019. "Evaluation of cost-effective building retrofit strategies through soft-linking a metamodel-based Bayesian method and a life cycle cost assessment method," Applied Energy, Elsevier, vol. 253(C), pages 1-1.
    3. Yuan, Jun & Nian, Victor & Su, Bin & Meng, Qun, 2017. "A simultaneous calibration and parameter ranking method for building energy models," Applied Energy, Elsevier, vol. 206(C), pages 657-666.
    4. Neves Costa, João & Ambrósio, Jorge & Andrade, António R. & Frey, Daniel, 2023. "Safety assessment using computer experiments and surrogate modeling: Railway vehicle safety and track quality indices," Reliability Engineering and System Safety, Elsevier, vol. 229(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kim, Wongon & Yoon, Heonjun & Lee, Guesuk & Kim, Taejin & Youn, Byeng D., 2020. "A new calibration metric that considers statistical correlation: Marginal Probability and Correlation Residuals," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    2. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    3. Perrin, G., 2020. "Adaptive calibration of a computer code with time-series output," Reliability Engineering and System Safety, Elsevier, vol. 196(C).
    4. I. Andrianakis & I. Vernon & N. McCreesh & T. J. McKinley & J. E. Oakley & R. N. Nsubuga & M. Goldstein & R. G. White, 2017. "History matching of a complex epidemiological model of human immunodeficiency virus transmission by using variance emulation," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 66(4), pages 717-740, August.
    5. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    6. Ioannis Andrianakis & Ian R Vernon & Nicky McCreesh & Trevelyan J McKinley & Jeremy E Oakley & Rebecca N Nsubuga & Michael Goldstein & Richard G White, 2015. "Bayesian History Matching of Complex Infectious Disease Models Using Emulation: A Tutorial and a Case Study on HIV in Uganda," PLOS Computational Biology, Public Library of Science, vol. 11(1), pages 1-18, January.
    7. Mevin Hooten & Christopher Wikle & Michael Schwob, 2020. "Statistical Implementations of Agent‐Based Demographic Models," International Statistical Review, International Statistical Institute, vol. 88(2), pages 441-461, August.
    8. Leatherman, Erin R. & Dean, Angela M. & Santner, Thomas J., 2017. "Designing combined physical and computer experiments to maximize prediction accuracy," Computational Statistics & Data Analysis, Elsevier, vol. 113(C), pages 346-362.
    9. Wu, Xu & Kozlowski, Tomasz & Meidani, Hadi, 2018. "Kriging-based inverse uncertainty quantification of nuclear fuel performance code BISON fission gas release model using time series measurement data," Reliability Engineering and System Safety, Elsevier, vol. 169(C), pages 422-436.
    10. Hjelkrem, Anne-Grete Roer & Höglind, Mats & van Oijen, Marcel & Schellberg, Jürgen & Gaiser, Thomas & Ewert, Frank, 2017. "Sensitivity analysis and Bayesian calibration for testing robustness of the BASGRA model in different environments," Ecological Modelling, Elsevier, vol. 359(C), pages 80-91.
    11. D. A. Henderson & R. J. Boys & D. J. Wilkinson, 2010. "Bayesian Calibration of a Stochastic Kinetic Computer Model Using Multiple Data Sources," Biometrics, The International Biometric Society, vol. 66(1), pages 249-256, March.
    12. Wu, Danqing & Lu, Zhenzhou & Wang, Yanping & Cheng, Lei, 2015. "Model validation and calibration based on component functions of model output," Reliability Engineering and System Safety, Elsevier, vol. 140(C), pages 59-70.
    13. Wilkinson Richard David, 2013. "Approximate Bayesian computation (ABC) gives exact results under the assumption of model error," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 12(2), pages 129-141, May.
    14. Antony M. Overstall & David C. Woods, 2013. "A Strategy for Bayesian Inference for Computationally Expensive Models with Application to the Estimation of Stem Cell Properties," Biometrics, The International Biometric Society, vol. 69(2), pages 458-468, June.
    15. Park, Inseok & Grandhi, Ramana V., 2014. "A Bayesian statistical method for quantifying model form uncertainty and two model combination methods," Reliability Engineering and System Safety, Elsevier, vol. 129(C), pages 46-56.
    16. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    17. Ehsan Mehdad & Jack P. C. Kleijnen, 2018. "Efficient global optimisation for black-box simulation via sequential intrinsic Kriging," Journal of the Operational Research Society, Taylor & Francis Journals, vol. 69(11), pages 1725-1737, November.
    18. Matthias Katzfuss & Joseph Guinness & Wenlong Gong & Daniel Zilber, 2020. "Vecchia Approximations of Gaussian-Process Predictions," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(3), pages 383-414, September.
    19. Ehsan Mehdad & Jack P.C. Kleijnen, 2018. "Stochastic intrinsic Kriging for simulation metamodeling," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 34(3), pages 322-337, May.
    20. Jakub Bijak & Jason D. Hilton & Eric Silverman & Viet Dung Cao, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:111:y:2013:i:c:p:273-286. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.