IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v34y2022i4p2368-2382.html
   My bibliography  Save this article

Reducing and Calibrating for Input Model Bias in Computer Simulation

Author

Listed:
  • Lucy E. Morgan

    (Department of Management Science, Lancaster University, Lancaster LA1 4YR, United Kingdom)

  • Luke Rhodes-Leader

    (Statistics and Operational Research Centre for Doctoral Training in Partnership with Industry (STOR-i), Lancaster University, Lancaster LA1 4YR, United Kingdom)

  • Russell R. Barton

    (Department of Supply Chain and Information Systems, The Pennsylvania State University, University Park, Pennsylvania 16802)

Abstract

Input model bias is the bias found in the output performance measures of a simulation model caused by estimating the input distributions/processes used to drive it. When the simulation response is a nonlinear function of its inputs, as is usually the case when simulating complex systems, input modelling bias is amongst the errors that arise. In this paper, we introduce a method that recalibrates the input parameters of parametric input models to reduce the bias in the simulation output. The proposed method is based on sequential quadratic programming with a closed form analytical solution at each step. An algorithm with guidance on how to practically implement the method is presented. The method is shown to be successful in reducing input modelling bias and the total mean squared error caused by input modelling error. Summary of Contribution: This paper furthers the understanding and treatment of input modelling error in computer simulation. We provide a novel method for reducing input model bias by recalibrating the input parameters used to drive a simulation model. A sequential quadratic programming approach with an explicit solution is provided to recalibrate the input parameters. The method is therefore computationally inexpensive. An algorithm outlining our proposed procedure is provided within the paper. An evaluation of the method shows the method successfully reduces input model bias and may also reduce the mean squared error caused by input modelling in the output of a simulation model.

Suggested Citation

  • Lucy E. Morgan & Luke Rhodes-Leader & Russell R. Barton, 2022. "Reducing and Calibrating for Input Model Bias in Computer Simulation," INFORMS Journal on Computing, INFORMS, vol. 34(4), pages 2368-2382, July.
    • Handle: RePEc:inm:orijoc:v:34:y:2022:i:4:p:2368-2382
    DOI: 10.1287/ijoc.2022.1183
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2022.1183
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2022.1183?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Morgan, Lucy E. & Nelson, Barry L. & Titman, Andrew C. & Worthington, David J., 2019. "Detecting bias due to input modelling in computer simulation," European Journal of Operational Research, Elsevier, vol. 279(3), pages 869-881.
    2. Saurabh Bansal & Genaro J. Gutierrez & John R. Keiser, 2017. "Using Experts’ Noisy Quantile Judgments to Quantify Risks: Theory and Application to Agribusiness," Operations Research, INFORMS, vol. 65(5), pages 1115-1130, October.
    3. Russell C. H. Cheng & Jack P. C. Kleijnen, 1999. "Improved Design of Queueing Simulation Experiments with Highly Heteroscedastic Responses," Operations Research, INFORMS, vol. 47(5), pages 762-777, October.
    4. Y Lin & E Song & B L Nelson, 2015. "Single-experiment input uncertainty," Journal of Simulation, Taylor & Francis Journals, vol. 9(3), pages 249-259, August.
    5. Barry L. Nelson & Alan T. K. Wan & Guohua Zou & Xinyu Zhang & Xi Jiang, 2021. "Reducing Simulation Input-Model Risk via Input Model Averaging," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 672-684, May.
    6. Marc C. Kennedy & Anthony O'Hagan, 2001. "Bayesian calibration of computer models," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(3), pages 425-464.
    7. Luís Felipe Bueno & Gabriel Haeser & Luiz-Rafael Santos, 2020. "Towards an efficient augmented Lagrangian method for convex quadratic programming," Computational Optimization and Applications, Springer, vol. 76(3), pages 767-800, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eunhye Song & Henry Lam & Russell R. Barton, 2024. "A Shrinkage Approach to Improve Direct Bootstrap Resampling Under Input Uncertainty," INFORMS Journal on Computing, INFORMS, vol. 36(4), pages 1023-1039, July.
    2. Vanslette, Kevin & Tohme, Tony & Youcef-Toumi, Kamal, 2020. "A general model validation and testing tool," Reliability Engineering and System Safety, Elsevier, vol. 195(C).
    3. Matthias Katzfuss & Joseph Guinness & Wenlong Gong & Daniel Zilber, 2020. "Vecchia Approximations of Gaussian-Process Predictions," Journal of Agricultural, Biological and Environmental Statistics, Springer;The International Biometric Society;American Statistical Association, vol. 25(3), pages 383-414, September.
    4. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    5. Jakub Bijak & Viet Dung Cao & Eric Silverman & Jason Hilton, 2013. "Reforging the Wedding Ring," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 29(27), pages 729-766.
    6. Hao Wu & Michael Browne, 2015. "Random Model Discrepancy: Interpretations and Technicalities (A Rejoinder)," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 619-624, September.
    7. Saurabh Bansal & Mahesh Nagarajan, 2017. "Product Portfolio Management with Production Flexibility in Agribusiness," Operations Research, INFORMS, vol. 65(4), pages 914-930, August.
    8. Villez, Kris & Del Giudice, Dario & Neumann, Marc B. & Rieckermann, Jörg, 2020. "Accounting for erroneous model structures in biokinetic process models," Reliability Engineering and System Safety, Elsevier, vol. 203(C).
    9. Xiaoyu Xiong & Benjamin D. Youngman & Theodoros Economou, 2021. "Data fusion with Gaussian processes for estimation of environmental hazard events," Environmetrics, John Wiley & Sons, Ltd., vol. 32(3), May.
    10. Petropoulos, G. & Wooster, M.J. & Carlson, T.N. & Kennedy, M.C. & Scholze, M., 2009. "A global Bayesian sensitivity analysis of the 1d SimSphere soil–vegetation–atmospheric transfer (SVAT) model using Gaussian model emulation," Ecological Modelling, Elsevier, vol. 220(19), pages 2427-2440.
    11. David Breitenmoser & Francesco Cerutti & Gernot Butterweck & Malgorzata Magdalena Kasprzak & Sabine Mayer, 2023. "Emulator-based Bayesian inference on non-proportional scintillation models by compton-edge probing," Nature Communications, Nature, vol. 14(1), pages 1-12, December.
    12. Drignei, Dorin, 2011. "A general statistical model for computer experiments with time series output," Reliability Engineering and System Safety, Elsevier, vol. 96(4), pages 460-467.
    13. Yuan, Jun & Nian, Victor & Su, Bin & Meng, Qun, 2017. "A simultaneous calibration and parameter ranking method for building energy models," Applied Energy, Elsevier, vol. 206(C), pages 657-666.
    14. Barde, Sylvain, 2024. "Bayesian estimation of large-scale simulation models with Gaussian process regression surrogates," Computational Statistics & Data Analysis, Elsevier, vol. 196(C).
    15. Gross, Eitan, 2015. "Effect of environmental stress on regulation of gene expression in the yeast," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 430(C), pages 224-235.
    16. Hwang, Youngdeok & Kim, Hang J. & Chang, Won & Yeo, Kyongmin & Kim, Yongku, 2019. "Bayesian pollution source identification via an inverse physics model," Computational Statistics & Data Analysis, Elsevier, vol. 134(C), pages 76-92.
    17. Choi, Wonjun & Menberg, Kathrin & Kikumoto, Hideki & Heo, Yeonsook & Choudhary, Ruchi & Ooka, Ryozo, 2018. "Bayesian inference of structural error in inverse models of thermal response tests," Applied Energy, Elsevier, vol. 228(C), pages 1473-1485.
    18. Yuan, Jun & Ng, Szu Hui, 2013. "A sequential approach for stochastic computer model calibration and prediction," Reliability Engineering and System Safety, Elsevier, vol. 111(C), pages 273-286.
    19. Edward Boone & Jan Hannig & Ryad Ghanam & Sujit Ghosh & Fabrizio Ruggeri & Serge Prudhomme, 2022. "Model Validation of a Single Degree-of-Freedom Oscillator: A Case Study," Stats, MDPI, vol. 5(4), pages 1-17, November.
    20. Jason R. W. Merrick & Claire A. Dorsey & Bo Wang & Martha Grabowski & John R. Harrald, 2022. "Measuring Prediction Accuracy in a Maritime Accident Warning System," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 819-827, February.

    More about this item

    Keywords

    ;
    ;
    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:34:y:2022:i:4:p:2368-2382. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.