IDEAS home Printed from
   My bibliography  Save this article

A methodology for fitting and validating metamodels in simulation


  • Kleijnen, Jack P. C.
  • Sargent, Robert G.


This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • Kleijnen, Jack P. C. & Sargent, Robert G., 2000. "A methodology for fitting and validating metamodels in simulation," European Journal of Operational Research, Elsevier, vol. 120(1), pages 14-29, January.
  • Handle: RePEc:eee:ejores:v:120:y:2000:i:1:p:14-29

    Download full text from publisher

    File URL:
    Download Restriction: Full text for ScienceDirect subscribers only

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    1. Kleijnen, J.P.C., 1995. "Sensitivity analysis and optimization of system dynamics models : Regression analysis and statistical design of experiments," Discussion Paper 1995-4, Tilburg University, Center for Economic Research.
    2. Kleijnen, Jack P. C. & Standridge, Charles R., 1988. "Experimental design and regression analysis in simulation: An FMS case study," European Journal of Operational Research, Elsevier, vol. 33(3), pages 257-261, February.
    3. Jack P. C. Kleijnen & Bert Bettonvil & Willem Van Groenendaal, 1998. "Validation of Trace-Driven Simulation Models: A Novel Regression Test," Management Science, INFORMS, vol. 44(6), pages 812-819, June.
    4. Kleijnen, Jack P. C., 1995. "Verification and validation of simulation models," European Journal of Operational Research, Elsevier, vol. 82(1), pages 145-162, April.
    5. van Groenendaal, W.J.H. & Kleijnen, J.P.C., 1997. "On the assessment of economic risk : Factorial design versus Monte Carlo methods," Other publications TiSEM fd2a2307-0812-4543-8151-7, Tilburg University, School of Economics and Management.
    6. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    7. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
    8. Jack P. C. Kleijnen, 1992. "Regression Metamodels for Simulation with Common Random Numbers: Comparison of Validation Tests and Confidence Intervals," Management Science, INFORMS, vol. 38(8), pages 1164-1185, August.
    9. Saltelli, A. & Andres, T. H. & Homma, T., 1995. "Sensitivity analysis of model output. Performance of the iterated fractional factorial design method," Computational Statistics & Data Analysis, Elsevier, vol. 20(4), pages 387-407, October.
    10. van Ham, G. & Rotmans, J. & Kleijnen, J.P.C., 1992. "Techniques for sensitivity analysis of simulation models : A case study of the CO2 greenhouse effect," Other publications TiSEM 71317a03-3399-4554-83cb-4, Tilburg University, School of Economics and Management.
    Full references (including those not matched with items on IDEAS)

    More about this item


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:120:y:2000:i:1:p:14-29. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Dana Niculescu). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.