IDEAS home Printed from https://ideas.repec.org/a/inm/orijoc/v34y2022i4p2368-2382.html
   My bibliography  Save this article

Reducing and Calibrating for Input Model Bias in Computer Simulation

Author

Listed:
  • Lucy E. Morgan

    (Department of Management Science, Lancaster University, Lancaster LA1 4YR, United Kingdom)

  • Luke Rhodes-Leader

    (Statistics and Operational Research Centre for Doctoral Training in Partnership with Industry (STOR-i), Lancaster University, Lancaster LA1 4YR, United Kingdom)

  • Russell R. Barton

    (Department of Supply Chain and Information Systems, The Pennsylvania State University, University Park, Pennsylvania 16802)

Abstract

Input model bias is the bias found in the output performance measures of a simulation model caused by estimating the input distributions/processes used to drive it. When the simulation response is a nonlinear function of its inputs, as is usually the case when simulating complex systems, input modelling bias is amongst the errors that arise. In this paper, we introduce a method that recalibrates the input parameters of parametric input models to reduce the bias in the simulation output. The proposed method is based on sequential quadratic programming with a closed form analytical solution at each step. An algorithm with guidance on how to practically implement the method is presented. The method is shown to be successful in reducing input modelling bias and the total mean squared error caused by input modelling error. Summary of Contribution: This paper furthers the understanding and treatment of input modelling error in computer simulation. We provide a novel method for reducing input model bias by recalibrating the input parameters used to drive a simulation model. A sequential quadratic programming approach with an explicit solution is provided to recalibrate the input parameters. The method is therefore computationally inexpensive. An algorithm outlining our proposed procedure is provided within the paper. An evaluation of the method shows the method successfully reduces input model bias and may also reduce the mean squared error caused by input modelling in the output of a simulation model.

Suggested Citation

  • Lucy E. Morgan & Luke Rhodes-Leader & Russell R. Barton, 2022. "Reducing and Calibrating for Input Model Bias in Computer Simulation," INFORMS Journal on Computing, INFORMS, vol. 34(4), pages 2368-2382, July.
    • Handle: RePEc:inm:orijoc:v:34:y:2022:i:4:p:2368-2382
    DOI: 10.1287/ijoc.2022.1183
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/ijoc.2022.1183
    Download Restriction: no
    File URL: https://libkey.io/10.1287/ijoc.2022.1183?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Morgan, Lucy E. & Nelson, Barry L. & Titman, Andrew C. & Worthington, David J., 2019. "Detecting bias due to input modelling in computer simulation," European Journal of Operational Research, Elsevier, vol. 279(3), pages 869-881.
    2. Saurabh Bansal & Genaro J. Gutierrez & John R. Keiser, 2017. "Using Experts’ Noisy Quantile Judgments to Quantify Risks: Theory and Application to Agribusiness," Operations Research, INFORMS, vol. 65(5), pages 1115-1130, October.
    3. Russell C. H. Cheng & Jack P. C. Kleijnen, 1999. "Improved Design of Queueing Simulation Experiments with Highly Heteroscedastic Responses," Operations Research, INFORMS, vol. 47(5), pages 762-777, October.
    4. Y Lin & E Song & B L Nelson, 2015. "Single-experiment input uncertainty," Journal of Simulation, Taylor & Francis Journals, vol. 9(3), pages 249-259, August.
    5. Barry L. Nelson & Alan T. K. Wan & Guohua Zou & Xinyu Zhang & Xi Jiang, 2021. "Reducing Simulation Input-Model Risk via Input Model Averaging," INFORMS Journal on Computing, INFORMS, vol. 33(2), pages 672-684, May.
    6. Luís Felipe Bueno & Gabriel Haeser & Luiz-Rafael Santos, 2020. "Towards an efficient augmented Lagrangian method for convex quadratic programming," Computational Optimization and Applications, Springer, vol. 76(3), pages 767-800, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    2. Saurabh Bansal & Mahesh Nagarajan, 2017. "Product Portfolio Management with Production Flexibility in Agribusiness," Operations Research, INFORMS, vol. 65(4), pages 914-930, August.
    3. Jason R. W. Merrick & Claire A. Dorsey & Bo Wang & Martha Grabowski & John R. Harrald, 2022. "Measuring Prediction Accuracy in a Maritime Accident Warning System," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 819-827, February.
    4. Jack P. C. Kleijnen & Susan M. Sanchez & Thomas W. Lucas & Thomas M. Cioppa, 2005. "State-of-the-Art Review: A User’s Guide to the Brave New World of Designing Simulation Experiments," INFORMS Journal on Computing, INFORMS, vol. 17(3), pages 263-289, August.
    5. Ernesto G. Birgin, 2020. "Preface of the special issue dedicated to the XII Brazilian workshop on continuous optimization," Computational Optimization and Applications, Springer, vol. 76(3), pages 615-619, July.
    6. Kleijnen, J.P.C. & Sanchez, S.M. & Lucas, T.W. & Cioppa, T.M., 2003. "A User's Guide to the Brave New World of Designing Simulation Experiments," Discussion Paper 2003-1, Tilburg University, Center for Economic Research.
    7. Feng Yang & Bruce E. Ankenman & Barry L. Nelson, 2008. "Estimating Cycle Time Percentile Curves for Manufacturing Systems via Simulation," INFORMS Journal on Computing, INFORMS, vol. 20(4), pages 628-643, November.
    8. Feng Yang & Bruce Ankenman & Barry L. Nelson, 2007. "Efficient generation of cycle time‐throughput curves through simulation and metamodeling," Naval Research Logistics (NRL), John Wiley & Sons, vol. 54(1), pages 78-93, February.
    9. Chidambaram Subbiah & Andrea C. Hupman & Haitao Li & Joseph Simonis, 2023. "Improving Software Development Effort Estimation with a Novel Design Pattern Model," Interfaces, INFORMS, vol. 53(3), pages 192-206, May.
    10. Yang, Feng & Liu, Jingang, 2012. "Simulation-based transfer function modeling for transient analysis of general queueing systems," European Journal of Operational Research, Elsevier, vol. 223(1), pages 150-166.
    11. Kleijnen, Jack P. C. & van Beers, Wim C. M., 2005. "Robustness of Kriging when interpolating in random simulation with heterogeneous variances: Some experiments," European Journal of Operational Research, Elsevier, vol. 165(3), pages 826-834, September.
    12. Saurabh Bansal & Genaro J. Gutierrez, 2020. "Estimating Uncertainties Using Judgmental Forecasts with Expert Heterogeneity," Operations Research, INFORMS, vol. 68(2), pages 363-380, March.
    13. Sulian Wang & Chen Wang, 2021. "Quantile Judgments of Lognormal Losses: An Experimental Investigation," Decision Analysis, INFORMS, vol. 18(1), pages 78-99, March.
    14. Saurabh Bansal & James S. Dyer, 2020. "Planning for End-User Substitution in Agribusiness," Operations Research, INFORMS, vol. 68(4), pages 1000-1019, July.
    15. Saurabh Bansal & Genaro J. Gutierrez & Mahesh Nagarajan, 2021. "Theory-Driven Practical Approach to Integrate R&D and Production Planning for Portfolio Management in Agribusiness," Interfaces, INFORMS, vol. 51(5), pages 332-346, September.
    16. Anil Gaba & Dana G. Popescu & Zhi Chen, 2019. "Assessing Uncertainty from Point Forecasts," Management Science, INFORMS, vol. 65(1), pages 90-106, January.
    17. Sungmin Park & John W. Fowler & Gerald T. Mackulak & J. Bert Keats & W. Matthew Carlyle, 2002. "D-Optimal Sequential Experiments for Generating a Simulation-Based Cycle Time-Throughput Curve," Operations Research, INFORMS, vol. 50(6), pages 981-990, December.
    18. Sigal Levy & David Steinberg, 2010. "Computer experiments: a review," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 94(4), pages 311-324, December.
    19. Saurabh Bansal & Yaroslav Rosokha, 2018. "Impact of Compound and Reduced Specification on Valuation of Projects with Multiple Risks," Decision Analysis, INFORMS, vol. 15(1), pages 27-46, March.
    20. Lance W. Saunders & J. Paul Brooks & Jason R. W. Merrick & Chad W. Autry, 2020. "Addressing Economic/Environmental Sustainability Trade‐offs in Procurement Episodes with Industrial Suppliers," Production and Operations Management, Production and Operations Management Society, vol. 29(5), pages 1256-1269, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:orijoc:v:34:y:2022:i:4:p:2368-2382. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.