Advanced Search
MyIDEAS: Login to save this article or follow this journal

A methodology for fitting and validating metamodels in simulation


Author Info

  • Kleijnen, Jack P. C.
  • Sargent, Robert G.


This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.

(This abstract was borrowed from another version of this item.)

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL:
Download Restriction: Full text for ScienceDirect subscribers only

As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.

Bibliographic Info

Article provided by Elsevier in its journal European Journal of Operational Research.

Volume (Year): 120 (2000)
Issue (Month): 1 (January)
Pages: 14-29

as in new window
Handle: RePEc:eee:ejores:v:120:y:2000:i:1:p:14-29

Contact details of provider:
Web page:

Related research


Other versions of this item:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Kleijnen, J.P.C., 1992. "Regression metamodels for simulation with common random numbers: Comparison of validation tests and confidence intervals," Open Access publications from Tilburg University urn:nbn:nl:ui:12-369802, Tilburg University.
  2. Kleijnen, J.P.C., 1995. "Case study: Statistical validation of simulation models," Open Access publications from Tilburg University urn:nbn:nl:ui:12-369796, Tilburg University.
  3. Kleijnen, J.P.C., 1992. "Verification and validation of simulation models," Research Memorandum 542, Tilburg University, Faculty of Economics and Business Administration.
Full references (including those not matched with items on IDEAS)


Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Galelli, S. & Gandolfi, C. & Soncini-Sessa, R. & Agostani, D., 2010. "Building a metamodel of an irrigation district distributed-parameter model," Agricultural Water Management, Elsevier, vol. 97(2), pages 187-200, February.
  2. Batmaz, Inci & Tunali, Semra, 2003. "Small response surface designs for metamodel estimation," European Journal of Operational Research, Elsevier, vol. 145(2), pages 455-470, March.
  3. Noguera, Jose H. & Watson, Edward F., 2006. "Response surface analysis of a multi-product batch processing facility using a simulation metamodel," International Journal of Production Economics, Elsevier, vol. 102(2), pages 333-343, August.
  4. Kleijnen, Jack P.C. & Mehdad, E., 2012. "Kriging in Multi-response Simulation, including a Monte Carlo Laboratory (Replaced by 2014-012)," Discussion Paper 2012-039, Tilburg University, Center for Economic Research.
  5. Ekren, Orhan & Ekren, Banu Yetkin, 2008. "Size optimization of a PV/wind hybrid energy conversion system with battery storage using response surface methodology," Applied Energy, Elsevier, vol. 85(11), pages 1086-1101, November.
  6. Reis dos Santos, Pedro M. & Isabel Reis dos Santos, M., 2009. "Using subsystem linear regression metamodels in stochastic simulation," European Journal of Operational Research, Elsevier, vol. 196(3), pages 1031-1040, August.
  7. Husslage, B.G.M. & Dam, E.R. van & Hertog, D. den & Stehouwer, H.P. & Stinstra, E., 2003. "Coordination of Coupled Black Box Simulations in the Construction of Metamodels," Discussion Paper 2003-2, Tilburg University, Center for Economic Research.
  8. Durieux, Severine & Pierreval, Henri, 2004. "Regression metamodeling for the design of automated manufacturing system composed of parallel machines sharing a material handling resource," International Journal of Production Economics, Elsevier, vol. 89(1), pages 21-30, May.
  9. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
  10. Marjolijn Haasnoot & Hans Middelkoop & Astrid Offermans & Eelco Beek & Willem Deursen, 2012. "Exploring pathways for sustainable water management in river deltas in a changing environment," Climatic Change, Springer, vol. 115(3), pages 795-819, December.
  11. van der Gaag, Monique A. & Vos, Fred & Saatkamp, Helmut W. & van Boven, Michiel & van Beek, Paul & Huirne, Ruud B. M., 2004. "A state-transition simulation model for the spread of Salmonella in the pork supply chain," European Journal of Operational Research, Elsevier, vol. 156(3), pages 782-798, August.
  12. Wise, Russell M. & Cacho, Oscar J., 2006. "Optimal Land-Use Decisions in the Presence of Carbon Payments and Fertilizer Subsidies: An Indonesian Case Study," 2006 Annual Meeting, August 12-18, 2006, Queensland, Australia 25356, International Association of Agricultural Economists.
  13. Vonk Noordegraaf, Antonie & Nielen, Mirjam & Kleijnen, Jack P. C., 2003. "Sensitivity analysis by experimental design and metamodelling: Case study on simulation in national animal disease control," European Journal of Operational Research, Elsevier, vol. 146(3), pages 433-443, May.
  14. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
  15. Tunali, S. & Batmaz, I., 2003. "A metamodeling methodology involving both qualitative and quantitative input factors," European Journal of Operational Research, Elsevier, vol. 150(2), pages 437-450, October.
  16. Marrel, Amandine & Iooss, Bertrand & Van Dorpe, Fran├žois & Volkova, Elena, 2008. "An efficient methodology for modeling complex computer codes with Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4731-4744, June.
  17. Graham, Tennille, 2005. "On the Road to Better Management: An investigation into the benefits of managing the impacts of dryland salinity on roads," 2005 Conference (49th), February 9-11, 2005, Coff's Harbour, Australia 137921, Australian Agricultural and Resource Economics Society.
  18. Ekren, Orhan & Ekren, Banu Y. & Ozerdem, Baris, 2009. "Break-even analysis and size optimization of a PV/wind hybrid energy conversion system with battery storage - A case study," Applied Energy, Elsevier, vol. 86(7-8), pages 1043-1054, July.
  19. Poropudas, Jirka & Virtanen, Kai, 2011. "Simulation metamodeling with dynamic Bayesian networks," European Journal of Operational Research, Elsevier, vol. 214(3), pages 644-655, November.


This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.


Access and download statistics


When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:120:y:2000:i:1:p:14-29. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.