IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v277y2019i1p300-316.html
   My bibliography  Save this article

Bayesian sequential data collection for stochastic simulation calibration

Author

Listed:
  • Wang, Bo
  • Zhang, Qiong
  • Xie, Wei

Abstract

Simulation is often used to guide the decision making for real complex stochastic systems. To faithfully assess the mean performance of the real system, it is necessary to efficiently calibrate the simulation model. Existing calibration approaches are typically built on the summary statistics of simulation outputs and ignore the serial dependence of detailed output sample paths. Given a tight simulation budget, we develop a Bayesian sequential data collection approach for simulation calibration via exploring the detailed simulation outputs. Then, the calibrated simulation model can be used to guide decision making. Both theoretical and empirical studies demonstrate that we can efficiently use the simulation resources and achieve better calibration accuracy by exploring the first two moment dynamic information of simulation output sample paths.

Suggested Citation

  • Wang, Bo & Zhang, Qiong & Xie, Wei, 2019. "Bayesian sequential data collection for stochastic simulation calibration," European Journal of Operational Research, Elsevier, vol. 277(1), pages 300-316.
    • Handle: RePEc:eee:ejores:v:277:y:2019:i:1:p:300-316
    DOI: 10.1016/j.ejor.2019.01.073
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221719301109
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ejor.2019.01.073?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Raymond K. W. Wong & Curtis B. Storlie & Thomas C. M. Lee, 2017. "A frequentist approach to computer model calibration," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(2), pages 635-648, March.
    2. Noel Cressie & Gardar Johannesson, 2008. "Fixed rank kriging for very large spatial data sets," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 70(1), pages 209-226, February.
    3. Huashuai Qu & Ilya O. Ryzhov & Michael C. Fu & Zi Ding, 2015. "Sequential Selection with Unknown Correlation Structures," Operations Research, INFORMS, vol. 63(4), pages 931-948, August.
    4. Matthew Plumlee, 2017. "Bayesian Calibration of Inexact Computer Models," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 1274-1285, July.
    5. Jun Luo & L. Jeff Hong & Barry L. Nelson & Yang Wu, 2015. "Fully Sequential Procedures for Large-Scale Ranking-and-Selection Problems in Parallel Computing Environments," Operations Research, INFORMS, vol. 63(5), pages 1177-1194, October.
    6. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    7. Ilya O. Ryzhov, 2016. "On the Convergence Rates of Expected Improvement Methods," Operations Research, INFORMS, vol. 64(6), pages 1515-1528, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Matthias Katzfuss & Joseph Guinness & Wenlong Gong & Daniel Zilber, 2020. "Vecchia Approximations of Gaussian-Process Predictions," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(3), pages 383-414, September.
    2. Na, Wei & Wang, Mingming, 2022. "A Bayesian approach with urban-scale energy model to calibrate building energy consumption for space heating: A case study of application in Beijing," Energy, Elsevier, vol. 247(C).
    3. Jun Yuan & Haowei Wang & Szu Hui Ng & Victor Nian, 2020. "Ship Emission Mitigation Strategies Choice Under Uncertainty," Energies, MDPI, vol. 13(9), pages 1-20, May.
    4. Saeid Delshad & Amin Khademi, 2020. "Information theory for ranking and selection," Naval Research Logistics (NRL), John Wiley & Sons, vol. 67(4), pages 239-253, June.
    5. Matthew Plumlee, 2014. "Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(508), pages 1581-1591, December.
    6. Chen, Yewen & Chang, Xiaohui & Luo, Fangzhi & Huang, Hui, 2023. "Additive dynamic models for correcting numerical model outputs," Computational Statistics & Data Analysis, Elsevier, vol. 187(C).
    7. Ye Chen & Ilya O. Ryzhov, 2020. "Technical Note—Consistency Analysis of Sequential Learning Under Approximate Bayesian Inference," Operations Research, INFORMS, vol. 68(1), pages 295-307, January.
    8. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    9. Zhongshun Shi & Yijie Peng & Leyuan Shi & Chun-Hung Chen & Michael C. Fu, 2022. "Dynamic Sampling Allocation Under Finite Simulation Budget for Feasibility Determination," INFORMS Journal on Computing, INFORMS, vol. 34(1), pages 557-568, January.
    10. Maupin, Kathryn A. & Swiler, Laura P., 2020. "Model discrepancy calibration across experimental settings," Reliability Engineering and System Safety, Elsevier, vol. 200(C).
    11. Wang, Tianxiang & Xu, Jie & Hu, Jian-Qiang & Chen, Chun-Hung, 2023. "Efficient estimation of a risk measure requiring two-stage simulation optimization," European Journal of Operational Research, Elsevier, vol. 305(3), pages 1355-1365.
    12. Benedict Götz & Sebastian Kersting & Michael Kohler, 2021. "Estimation of an improved surrogate model in uncertainty quantification by neural networks," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(2), pages 249-281, April.
    13. L. Jeff Hong & Guangxin Jiang & Ying Zhong, 2022. "Solving Large-Scale Fixed-Budget Ranking and Selection Problems," INFORMS Journal on Computing, INFORMS, vol. 34(6), pages 2930-2949, November.
    14. SungKu Kang & Ran Jin & Xinwei Deng & Ron S. Kenett, 2023. "Challenges of modeling and analysis in cybermanufacturing: a review from a machine learning and computation perspective," Journal of Intelligent Manufacturing, Springer, vol. 34(2), pages 415-428, February.
    15. Huaimin Diao & Yan Wang & Dianpeng Wang, 2022. "A D-Optimal Sequential Calibration Design for Computer Models," Mathematics, MDPI, vol. 10(9), pages 1-15, April.
    16. Jung, Yongsu & Jo, Hwisang & Choo, Jeonghwan & Lee, Ikjin, 2022. "Statistical model calibration and design optimization under aleatory and epistemic uncertainty," Reliability Engineering and System Safety, Elsevier, vol. 222(C).
    17. K. Shuvo Bakar & Nicholas Biddle & Philip Kokic & Huidong Jin, 2020. "A Bayesian spatial categorical model for prediction to overlapping geographical areas in sample surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 535-563, February.
    18. Hao Wu & Michael Browne, 2015. "Random Model Discrepancy: Interpretations and Technicalities (A Rejoinder)," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 619-624, September.
    19. Xiaoyu Xiong & Benjamin D. Youngman & Theodoros Economou, 2021. "Data fusion with Gaussian processes for estimation of environmental hazard events," Environmetrics, John Wiley & Sons, Ltd., vol. 32(3), May.
    20. Petropoulos, G. & Wooster, M.J. & Carlson, T.N. & Kennedy, M.C. & Scholze, M., 2009. "A global Bayesian sensitivity analysis of the 1d SimSphere soil–vegetation–atmospheric transfer (SVAT) model using Gaussian model emulation," Ecological Modelling, Elsevier, vol. 220(19), pages 2427-2440.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:277:y:2019:i:1:p:300-316. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.