A methodology for fitting and validating metamodels in simulation
This expository paper discusses the relationships among metamodels, simulation models, and problem entities. A metamodel or response surface is an approximation of the input/output function implied by the underlying simulation model. There are several types of metamodel: linear regression, splines, neural networks, etc. This paper distinguishes between fitting and validating a metamodel. Metamodels may have different goals: (i) understanding, (ii) prediction, (iii) optimization, and (iv) verification and validation. For this metamodeling, a process with thirteen steps is proposed. Classic design of experiments (DOE) is summarized, including standard measures of fit such as the R-square coefficient and cross-validation measures. This DOE is extended to sequential or stagewise DOE. Several validation criteria, measures, and estimators are discussed. Metamodels in general are covered, along with a procedure for developing linear regression (including polynomial) metamodels.
(This abstract was borrowed from another version of this item.)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January.
- van Ham, G. & Rotmans, J. & Kleijnen, J.P.C., 1992. "Techniques for sensitivity analysis of simulation models : A case study of the CO2 greenhouse effect," Other publications TiSEM 71317a03-3399-4554-83cb-4, Tilburg University, School of Economics and Management.
- Jack P. C. Kleijnen, 1992. "Regression Metamodels for Simulation with Common Random Numbers: Comparison of Validation Tests and Confidence Intervals," Management Science, INFORMS, vol. 38(8), pages 1164-1185, August.
- van Groenendaal, W.J.H. & Kleijnen, J.P.C., 1997. "On the assessment of economic risk : Factorial design versus Monte Carlo methods," Other publications TiSEM fd2a2307-0812-4543-8151-7, Tilburg University, School of Economics and Management.
- Francis X. Diebold & Robert S. Mariano, 1994.
"Comparing Predictive Accuracy,"
NBER Technical Working Papers
0169, National Bureau of Economic Research, Inc.
- Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-44, January.
- Diebold, Francis X & Mariano, Roberto S, 1995. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(3), pages 253-63, July.
- Kleijnen, J.P.C., 1992.
"Verification and validation of simulation models,"
FEW 542, Tilburg University, School of Economics and Management.
- Kleijnen, J.P.C., 1995. "Sensitivity analysis and optimization of system dynamics models : Regression analysis and statistical design of experiments," Discussion Paper 1995-4, Tilburg University, Center for Economic Research.
- Jack P. C. Kleijnen & Bert Bettonvil & Willem Van Groenendaal, 1998. "Validation of Trace-Driven Simulation Models: A Novel Regression Test," Management Science, INFORMS, vol. 44(6), pages 812-819, June.
- Kleijnen, J.P.C. & Standridge, C.R., 1986.
"Experimental design and regression analysis in simulation : An FMS case study,"
FEW 212, Tilburg University, School of Economics and Management.
- Kleijnen, Jack P. C. & Standridge, Charles R., 1988. "Experimental design and regression analysis in simulation: An FMS case study," European Journal of Operational Research, Elsevier, vol. 33(3), pages 257-261, February.
- Saltelli, A. & Andres, T. H. & Homma, T., 1995. "Sensitivity analysis of model output. Performance of the iterated fractional factorial design method," Computational Statistics & Data Analysis, Elsevier, vol. 20(4), pages 387-407, October.
When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:120:y:2000:i:1:p:14-29. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Shamier, Wendy)
If references are entirely missing, you can add them using this form.