Advanced Search
MyIDEAS: Login to save this paper or follow this series

A Methodology for Fitting and Validating Metamodels in Simulation


Author Info

  • Kleijnen, J.P.C.
  • Sargent, R.

    (Tilburg University, Center for Economic Research)


This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.

Download Info

If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
File URL:
Our checks indicate that this address may not be valid because: 404 Not Found. If this is indeed the case, please notify (Richard Broekman)
Download Restriction: no

Bibliographic Info

Paper provided by Tilburg University, Center for Economic Research in its series Discussion Paper with number 1997-116.

as in new window
Date of creation: 1997
Date of revision:
Handle: RePEc:dgr:kubcen:1997116

Contact details of provider:
Web page:

Related research

Keywords: Simulation; approximation; response surface; modelling; regression;

Other versions of this item:


References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
as in new window
  1. Kleijnen, J.P.C., 1992. "Regression metamodels for simulation with common random numbers: Comparison of validation tests and confidence intervals," Open Access publications from Tilburg University urn:nbn:nl:ui:12-369802, Tilburg University.
  2. Kleijnen, J.P.C., 1992. "Verification and validation of simulation models," Research Memorandum 542, Tilburg University, Faculty of Economics and Business Administration.
  3. Kleijnen, J.P.C., 1995. "Case study: Statistical validation of simulation models," Open Access publications from Tilburg University urn:nbn:nl:ui:12-369796, Tilburg University.
Full references (including those not matched with items on IDEAS)


Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
as in new window

Cited by:
  1. Ekren, Orhan & Ekren, Banu Y. & Ozerdem, Baris, 2009. "Break-even analysis and size optimization of a PV/wind hybrid energy conversion system with battery storage - A case study," Applied Energy, Elsevier, vol. 86(7-8), pages 1043-1054, July.
  2. Durieux, Severine & Pierreval, Henri, 2004. "Regression metamodeling for the design of automated manufacturing system composed of parallel machines sharing a material handling resource," International Journal of Production Economics, Elsevier, vol. 89(1), pages 21-30, May.
  3. Shi, Wen & Shang, Jennifer & Liu, Zhixue & Zuo, Xiaolu, 2014. "Optimal design of the auto parts supply chain for JIT operations: Sequential bifurcation factor screening and multi-response surface methodology," European Journal of Operational Research, Elsevier, vol. 236(2), pages 664-676.
  4. Reis dos Santos, Pedro M. & Isabel Reis dos Santos, M., 2009. "Using subsystem linear regression metamodels in stochastic simulation," European Journal of Operational Research, Elsevier, vol. 196(3), pages 1031-1040, August.
  5. Galelli, S. & Gandolfi, C. & Soncini-Sessa, R. & Agostani, D., 2010. "Building a metamodel of an irrigation district distributed-parameter model," Agricultural Water Management, Elsevier, vol. 97(2), pages 187-200, February.
  6. Graham, Tennille, 2005. "On the Road to Better Management: An investigation into the benefits of managing the impacts of dryland salinity on roads," 2005 Conference (49th), February 9-11, 2005, Coff's Harbour, Australia 137921, Australian Agricultural and Resource Economics Society.
  7. Marjolijn Haasnoot & Hans Middelkoop & Astrid Offermans & Eelco Beek & Willem Deursen, 2012. "Exploring pathways for sustainable water management in river deltas in a changing environment," Climatic Change, Springer, vol. 115(3), pages 795-819, December.
  8. Vonk Noordegraaf, Antonie & Nielen, Mirjam & Kleijnen, Jack P. C., 2003. "Sensitivity analysis by experimental design and metamodelling: Case study on simulation in national animal disease control," European Journal of Operational Research, Elsevier, vol. 146(3), pages 433-443, May.
  9. Wise, Russell M. & Cacho, Oscar J., 2006. "Optimal Land-Use Decisions in the Presence of Carbon Payments and Fertilizer Subsidies: An Indonesian Case Study," 2006 Annual Meeting, August 12-18, 2006, Queensland, Australia 25356, International Association of Agricultural Economists.
  10. Ekren, Orhan & Ekren, Banu Yetkin, 2008. "Size optimization of a PV/wind hybrid energy conversion system with battery storage using response surface methodology," Applied Energy, Elsevier, vol. 85(11), pages 1086-1101, November.
  11. Kleijnen, Jack P.C. & Mehdad, E., 2012. "Kriging in Multi-response Simulation, including a Monte Carlo Laboratory (Replaced by 2014-012)," Discussion Paper 2012-039, Tilburg University, Center for Economic Research.
  12. Tunali, S. & Batmaz, I., 2003. "A metamodeling methodology involving both qualitative and quantitative input factors," European Journal of Operational Research, Elsevier, vol. 150(2), pages 437-450, October.
  13. Noguera, Jose H. & Watson, Edward F., 2006. "Response surface analysis of a multi-product batch processing facility using a simulation metamodel," International Journal of Production Economics, Elsevier, vol. 102(2), pages 333-343, August.
  14. Batmaz, Inci & Tunali, Semra, 2003. "Small response surface designs for metamodel estimation," European Journal of Operational Research, Elsevier, vol. 145(2), pages 455-470, March.
  15. Poropudas, Jirka & Virtanen, Kai, 2011. "Simulation metamodeling with dynamic Bayesian networks," European Journal of Operational Research, Elsevier, vol. 214(3), pages 644-655, November.
  16. Marrel, Amandine & Iooss, Bertrand & Van Dorpe, Fran├žois & Volkova, Elena, 2008. "An efficient methodology for modeling complex computer codes with Gaussian processes," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4731-4744, June.
  17. Husslage, B.G.M. & Dam, E.R. van & Hertog, D. den & Stehouwer, H.P. & Stinstra, E., 2003. "Coordination of Coupled Black Box Simulations in the Construction of Metamodels," Discussion Paper 2003-2, Tilburg University, Center for Economic Research.
  18. Strang, Kenneth David, 2012. "Importance of verifying queue model assumptions before planning with simulation software," European Journal of Operational Research, Elsevier, vol. 218(2), pages 493-504.
  19. van der Gaag, Monique A. & Vos, Fred & Saatkamp, Helmut W. & van Boven, Michiel & van Beek, Paul & Huirne, Ruud B. M., 2004. "A state-transition simulation model for the spread of Salmonella in the pork supply chain," European Journal of Operational Research, Elsevier, vol. 156(3), pages 782-798, August.


This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.


Access and download statistics


When requesting a correction, please mention this item's handle: RePEc:dgr:kubcen:1997116. See general information about how to correct material in RePEc.

For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Broekman).

If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

If references are entirely missing, you can add them using this form.

If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

Please note that corrections may take a couple of weeks to filter through the various RePEc services.