IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v256y2017i1p1-16.html
   My bibliography  Save this article

Regression and Kriging metamodels with their experimental designs in simulation: A review

Author

Listed:
  • Kleijnen, Jack P.C.

Abstract

This article reviews the design and analysis of simulation experiments. It focusses on analysis via two types of metamodel (surrogate. emulator); namely, low-order polynomial regression, and Kriging (or Gaussian process). The metamodel type determines the design of the simulation experiment, which determines the input combinations of the simulation model. For example, a first-order polynomial regression metamodel should use a “resolution-III”design, whereas Kriging may use “Latin hypercube sampling”. More generally, polynomials of first or second order may use resolution III, IV, V, or “central composite” designs. Before applying either regression or Kriging metamodeling, the many inputs of a realistic simulation model can be screened via “sequential bifurcation”. Optimization of the simulated system may use either a sequence of low-order polynomials—known as “response surface methodology” (RSM)—or Kriging models fitted through sequential designs—including “efficient global optimization” (EGO). Finally, “robust”optimization accounts for uncertainty in some simulation inputs.

Suggested Citation

  • Kleijnen, Jack P.C., 2017. "Regression and Kriging metamodels with their experimental designs in simulation: A review," European Journal of Operational Research, Elsevier, vol. 256(1), pages 1-16.
  • Handle: RePEc:eee:ejores:v:256:y:2017:i:1:p:1-16
    DOI: 10.1016/j.ejor.2016.06.041
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221716304623
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2016.06.041?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kleijnen, Jack P.C. & Deflandre, David, 2006. "Validation of regression metamodels in simulation: Bootstrap approach," European Journal of Operational Research, Elsevier, vol. 170(1), pages 120-131, April.
    2. (Various), 2005. "Book Reviews," The Energy Journal, International Association for Energy Economics, vol. 0(Number 2), pages 129-136.
    3. Kleijnen, Jack P. C., 2005. "An overview of the design and analysis of simulation experiments for sensitivity analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 287-300, July.
    4. Shi, Wen & Kleijnen, Jack P.C. & Liu, Zhixue, 2014. "Factor screening for simulation with multiple responses: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 237(1), pages 136-147.
    5. Wei, Pengfei & Lu, Zhenzhou & Song, Jingwen, 2015. "Variable importance analysis: A comprehensive review," Reliability Engineering and System Safety, Elsevier, vol. 142(C), pages 399-432.
    6. Hamed Jalali & Inneke Van Nieuwenhuyse, 2015. "Simulation optimization in inventory replenishment: a classification," IISE Transactions, Taylor & Francis Journals, vol. 47(11), pages 1217-1235, November.
    7. Kleijnen, Jack P.C., 2009. "Kriging metamodeling in simulation: A review," European Journal of Operational Research, Elsevier, vol. 192(3), pages 707-716, February.
    8. van Beers, Wim C.M. & Kleijnen, Jack P.C., 2008. "Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping," European Journal of Operational Research, Elsevier, vol. 186(3), pages 1099-1113, May.
    9. Kamiński, Bogumił, 2015. "A method for the updating of stochastic kriging metamodels," European Journal of Operational Research, Elsevier, vol. 247(3), pages 859-866.
    10. Bettonvil, Bert & del Castillo, Enrique & Kleijnen, Jack P.C., 2009. "Statistical testing of optimality conditions in multiresponse simulation-based optimization," European Journal of Operational Research, Elsevier, vol. 199(2), pages 448-458, December.
    11. Michael B. Gordy & Sandeep Juneja, 2010. "Nested Simulation in Portfolio Risk Measurement," Management Science, INFORMS, vol. 56(10), pages 1833-1848, October.
    12. Unknown, 2005. "Review of National Competition Policy Reforms," Inquiry Reports 31898, Productivity Commission.
    13. Hong Wan & Bruce E. Ankenman & Barry L. Nelson, 2010. "Improving the Efficiency and Efficacy of Controlled Sequential Bifurcation for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 22(3), pages 482-492, August.
    14. Kleijnen, J.P.C. & van Beers, W.C.M. & van Nieuwenhuyse, I., 2008. "Constrained Optimization in Simulation : A Novel Approach," Other publications TiSEM e49ba0fc-853c-4a13-b564-d, Tilburg University, School of Economics and Management.
    15. (Various), 2005. "Book Reviews," The Energy Journal, International Association for Energy Economics, vol. 0(Number 3), pages 117-134.
    16. FitzGerald, John & Bergin, Adele & Kearney, Ide & Barrett, Alan & Duffy, David & Garrett, Shane & McCarthy, Yvonne, 2005. "Medium-Term Review 2005-2012, No. 10," Forecasting Report, Economic and Social Research Institute (ESRI), number MTR10, June.
    17. Kleijnen, Jack P. C. & Gaury, Eric, 2003. "Short-term robustness of production management systems: A case study," European Journal of Operational Research, Elsevier, vol. 148(2), pages 452-465, July.
    18. Kleijnen, Jack P.C. & Beers, Wim van & Nieuwenhuyse, Inneke van, 2010. "Constrained optimization in expensive simulation: Novel approach," European Journal of Operational Research, Elsevier, vol. 202(1), pages 164-174, April.
    19. Kleijnen, Jack P.C. & Pierreval, Henri & Zhang, Jin, 2011. "Methodology for determining the acceptability of system designs in uncertain environments," European Journal of Operational Research, Elsevier, vol. 209(2), pages 176-183, March.
    20. Crombecq, K. & Laermans, E. & Dhaene, T., 2011. "Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling," European Journal of Operational Research, Elsevier, vol. 214(3), pages 683-696, November.
    21. Unknown, 2005. "Book Reviews and Contents Pages," Journal of Rural Cooperation, Hebrew University, Center for Agricultural Economic Research, vol. 33(2), pages 1-9.
    22. Kuo-Hao Chang & Ming-Kai Li & Hong Wan, 2014. "Combining STRONG with screening designs for large-scale simulation optimization," IISE Transactions, Taylor & Francis Journals, vol. 46(4), pages 357-373.
    23. (Various), 2005. "Book Reviews," The Energy Journal, International Association for Energy Economics, vol. 0(Number 4).
    24. Borgonovo, Emanuele & Plischke, Elmar, 2016. "Sensitivity analysis: A review of recent advances," European Journal of Operational Research, Elsevier, vol. 248(3), pages 869-887.
    25. anonymous, 2005. "Guest Area Editors and Ad Hoc Reviewers," Marketing Science, INFORMS, vol. 24(1), pages 179-184.
    26. Bruce Ankenman & Barry L. Nelson & Jeremy Staum, 2010. "Stochastic Kriging for Simulation Metamodeling," Operations Research, INFORMS, vol. 58(2), pages 371-382, April.
    27. J P C Kleijnen & W C M van Beers, 2013. "Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 64(5), pages 708-717, May.
    28. Gabriella Dellino & Jack P. C. Kleijnen & Carlo Meloni, 2012. "Robust Optimization in Simulation: Taguchi and Krige Combined," INFORMS Journal on Computing, INFORMS, vol. 24(3), pages 471-484, August.
    29. Jack P.C. Kleijnen, 2015. "Design and Analysis of Simulation Experiments," International Series in Operations Research and Management Science, Springer, edition 2, number 978-3-319-18087-8, December.
    30. Bradley Efron, 2015. "Frequentist accuracy of Bayesian estimates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 77(3), pages 617-646, June.
    31. Markiewicz, A. & Szczepanska, A., 2007. "Optimal designs in multivariate linear models," Statistics & Probability Letters, Elsevier, vol. 77(4), pages 426-430, February.
    32. Wei Xie & Barry L. Nelson & Russell R. Barton, 2014. "A Bayesian Framework for Quantifying Uncertainty in Stochastic Simulation," Operations Research, INFORMS, vol. 62(6), pages 1439-1452, December.
    33. Unknown, 2005. "ARER Reviewers, September 2004 - August 2005," Agricultural and Resource Economics Review, Northeastern Agricultural and Resource Economics Association, vol. 34(2), pages 1-1, October.
    34. Helton, J.C. & Hansen, C.W. & Sallaberry, C.J., 2014. "Expected dose for the nominal scenario class in the 2008 performance assessment for the proposed high-level radioactive waste repository at Yucca Mountain, Nevada," Reliability Engineering and System Safety, Elsevier, vol. 122(C), pages 267-271.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Christian Troost & Julia Parussis-Krech & Matías Mejaíl & Thomas Berger, 2023. "Boosting the Scalability of Farm-Level Models: Efficient Surrogate Modeling of Compositional Simulation Output," Computational Economics, Springer;Society for Computational Economics, vol. 62(3), pages 721-759, October.
    2. Duro, João A. & Ozturk, Umud Esat & Oara, Daniel C. & Salomon, Shaul & Lygoe, Robert J. & Burke, Richard & Purshouse, Robin C., 2023. "Methods for constrained optimization of expensive mixed-integer multi-objective problems, with application to an internal combustion engine design problem," European Journal of Operational Research, Elsevier, vol. 307(1), pages 421-446.
    3. Tay, Timothy & Osorio, Carolina, 2022. "Bayesian optimization techniques for high-dimensional simulation-based transportation problems," Transportation Research Part B: Methodological, Elsevier, vol. 164(C), pages 210-243.
    4. Powell, Warren B., 2019. "A unified framework for stochastic optimization," European Journal of Operational Research, Elsevier, vol. 275(3), pages 795-821.
    5. J.-J. Sinou & L. Nechak & S. Besset, 2018. "Kriging Metamodeling in Rotordynamics: Application for Predicting Critical Speeds and Vibrations of a Flexible Rotor," Complexity, Hindawi, vol. 2018, pages 1-26, March.
    6. Liu, Yushan & Li, Luyi & Chang, Zeming, 2023. "Efficient Bayesian model updating for dynamic systems," Reliability Engineering and System Safety, Elsevier, vol. 236(C).
    7. Koziel, Slawomir & Pietrenko-Dabrowska, Anna, 2022. "Constrained multi-objective optimization of compact microwave circuits by design triangulation and pareto front interpolation," European Journal of Operational Research, Elsevier, vol. 299(1), pages 302-312.
    8. Na Li & Xiaorui Li & Paul Forero, 2022. "Physician scheduling for outpatient department with nonhomogeneous patient arrival and priority queue," Flexible Services and Manufacturing Journal, Springer, vol. 34(4), pages 879-915, December.
    9. Sylvain Barde & Sander van Der Hoog, 2017. "An empirical validation protocol for large-scale agent-based models," Working Papers hal-03458672, HAL.
    10. Pasta, Edoardo & Faedo, Nicolás & Mattiazzo, Giuliana & Ringwood, John V., 2023. "Towards data-driven and data-based control of wave energy systems: Classification, overview, and critical assessment," Renewable and Sustainable Energy Reviews, Elsevier, vol. 188(C).
    11. Capitanescu, F. & Marvuglia, A. & Benetto, E. & Ahmadi, A. & Tiruta-Barna, L., 2017. "Linear programming-based directed local search for expensive multi-objective optimization problems: Application to drinking water production plants," European Journal of Operational Research, Elsevier, vol. 262(1), pages 322-334.
    12. Xiangang Cao & Pengfei Li & Song Ming, 2021. "Remaining Useful Life Prediction-Based Maintenance Decision Model for Stochastic Deterioration Equipment under Data-Driven," Sustainability, MDPI, vol. 13(15), pages 1-19, July.
    13. Xie, Wei & Barton, Russell R. & Nelson, Barry L. & Wang, Keqi, 2023. "Stochastic simulation uncertainty analysis to accelerate flexible biomanufacturing process development," European Journal of Operational Research, Elsevier, vol. 310(1), pages 238-248.
    14. Song, Zhouzhou & Zhang, Hanyu & Liu, Zhao & Zhu, Ping, 2023. "A two-stage Kriging estimation variance reduction method for efficient time-variant reliability-based design optimization," Reliability Engineering and System Safety, Elsevier, vol. 237(C).
    15. Giulia Pedrielli & K. Selcuk Candan & Xilun Chen & Logan Mathesen & Alireza Inanalouganji & Jie Xu & Chun-Hung Chen & Loo Hay Lee, 2019. "Generalized Ordinal Learning Framework (GOLF) for Decision Making with Future Simulated Data," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 36(06), pages 1-35, December.
    16. Wu, Zeping & Wang, Wenjie & Wang, Donghui & Zhao, Kun & Zhang, Weihua, 2019. "Global sensitivity analysis using orthogonal augmented radial basis function," Reliability Engineering and System Safety, Elsevier, vol. 185(C), pages 291-302.
    17. Liu, Yushan & Li, Luyi & Zhao, Sihan & Song, Shufang, 2021. "A global surrogate model technique based on principal component analysis and Kriging for uncertainty propagation of dynamic systems," Reliability Engineering and System Safety, Elsevier, vol. 207(C).
    18. Miranda, Rafael de Carvalho & Montevechi, José Arnaldo Barra & da Silva, Aneirson Francisco & Marins, Fernando Augusto Silva, 2017. "Increasing the efficiency in integer simulation optimization: Reducing the search space through data envelopment analysis and orthogonal arrays," European Journal of Operational Research, Elsevier, vol. 262(2), pages 673-681.
    19. repec:hal:spmain:info:hdl:2441/4pa18fd9lf9h59m4vfavfcf61e is not listed on IDEAS
    20. Tang, Xinzi & Wang, Zhe & Xiao, Peng & Peng, Ruitao & Liu, Xiongwei, 2020. "Uncertainty quantification based optimization of centrifugal compressor impeller for aerodynamic robustness under stochastic operational conditions," Energy, Elsevier, vol. 195(C).
    21. Wang, Yabo & Yu, Jianyang & Song, Yanping & Chen, Fu, 2020. "Parameter optimization of the composite honeycomb tip in a turbine cascade," Energy, Elsevier, vol. 197(C).
    22. Li, Guosheng & Ma, Shuaichao & Zhang, Dequan & Yang, Leping & Zhang, Weihua & Wu, Zeping, 2024. "An efficient sequential anisotropic RBF reliability analysis method with fast cross-validation and parallelizability," Reliability Engineering and System Safety, Elsevier, vol. 241(C).
    23. Nuno Costa & Paulo Fontes, 2020. "Energy-Efficiency Assessment and Improvement—Experiments and Analysis Methods," Sustainability, MDPI, vol. 12(18), pages 1-19, September.
    24. Chen, Xi & Zhou, Qiang, 2017. "Sequential design strategies for mean response surface metamodeling via stochastic kriging with adaptive exploration and exploitation," European Journal of Operational Research, Elsevier, vol. 262(2), pages 575-585.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
    2. Kleijnen, Jack P.C., 2013. "Simulation-Optimization via Kriging and Bootstrapping : A Survey (Revision of CentER DP 2011-064)," Discussion Paper 2013-064, Tilburg University, Center for Economic Research.
    3. Minervini, Leo Fulvio, 2014. "Spectrum management reform: Rethinking practices," Telecommunications Policy, Elsevier, vol. 38(2), pages 136-146.
    4. Plischke, Elmar & Borgonovo, Emanuele & Smith, Curtis L., 2013. "Global sensitivity measures from given data," European Journal of Operational Research, Elsevier, vol. 226(3), pages 536-550.
    5. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    6. Jack P. C. Kleijnen, 2015. "Response Surface Methodology," International Series in Operations Research & Management Science, in: Michael C Fu (ed.), Handbook of Simulation Optimization, edition 127, chapter 0, pages 81-104, Springer.
    7. Ouyang, Linhan & Ma, Yizhong & Wang, Jianjun & Tu, Yiliu, 2017. "A new loss function for multi-response optimization with model parameter uncertainty and implementation errors," European Journal of Operational Research, Elsevier, vol. 258(2), pages 552-563.
    8. Scott L. Rosen & Christopher P. Saunders & Samar K Guharay, 2015. "A Structured Approach for Rapidly Mapping Multilevel System Measures via Simulation Metamodeling," Systems Engineering, John Wiley & Sons, vol. 18(1), pages 87-101, January.
    9. Wen Shi & Xi Chen, 2018. "Efficient budget allocation strategies for elementary effects method in stochastic simulation," Naval Research Logistics (NRL), John Wiley & Sons, vol. 65(3), pages 218-241, April.
    10. Ale Ebrahim, Nader & Ahmed Shamsuddin & Abdul Rashid, Salwa Hanim & Taha, Zahari, 2012. "Effective Virtual Teams for New Product Development," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 7(21), pages 1971-1985.
    11. Kleijnen, Jack P.C. & van Beers, W.C.M. & van Nieuwenhuyse, I., 2011. "Expected Improvement in Efficient Global Optimization Through Bootstrapped Kriging - Replaces CentER DP 2010-62," Discussion Paper 2011-015, Tilburg University, Center for Economic Research.
    12. Chen, Xi & Zhou, Qiang, 2017. "Sequential design strategies for mean response surface metamodeling via stochastic kriging with adaptive exploration and exploitation," European Journal of Operational Research, Elsevier, vol. 262(2), pages 575-585.
    13. Hernandez, Andres F. & Grover, Martha A., 2013. "Error estimation properties of Gaussian process models in stochastic simulations," European Journal of Operational Research, Elsevier, vol. 228(1), pages 131-140.
    14. Shi, Wen & Liu, Zhixue & Shang, Jennifer & Cui, Yujia, 2013. "Multi-criteria robust design of a JIT-based cross-docking distribution center for an auto parts supply chain," European Journal of Operational Research, Elsevier, vol. 229(3), pages 695-706.
    15. Xi Chen & Kyoung-Kuk Kim, 2016. "Efficient VaR and CVaR Measurement via Stochastic Kriging," INFORMS Journal on Computing, INFORMS, vol. 28(4), pages 629-644, November.
    16. Kabirian, Alireza & Ólafsson, Sigurdur, 2011. "Continuous optimization via simulation using Golden Region search," European Journal of Operational Research, Elsevier, vol. 208(1), pages 19-27, January.
    17. Kleijnen, Jack P.C. & Beers, Wim van & Nieuwenhuyse, Inneke van, 2010. "Constrained optimization in expensive simulation: Novel approach," European Journal of Operational Research, Elsevier, vol. 202(1), pages 164-174, April.
    18. Osorio, Carolina, 2019. "High-dimensional offline origin-destination (OD) demand calibration for stochastic traffic simulators of large-scale road networks," Transportation Research Part B: Methodological, Elsevier, vol. 124(C), pages 18-43.
    19. J P C Kleijnen & W C M van Beers, 2013. "Monotonicity-preserving bootstrapped Kriging metamodels for expensive simulations," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 64(5), pages 708-717, May.
    20. Helton, Jon C. & Brooks, Dusty M. & Sallaberry, Cédric J., 2020. "Margins associated with loss of assured safety for systems with multiple weak links and strong links," Reliability Engineering and System Safety, Elsevier, vol. 195(C).

    More about this item

    Keywords

    Robustness and sensitivity; Metamodel; Design; Regression; Kriging;
    All these keywords.

    JEL classification:

    • C0 - Mathematical and Quantitative Methods - - General
    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C15 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Statistical Simulation Methods: General
    • C44 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - Operations Research; Statistical Decision Theory

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:256:y:2017:i:1:p:1-16. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.