This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Experimental design for sensitivity analysis, optimization, and validation of simulation models

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Kleijnen, J.P.C. (Tilburg University, Center for Economic Research)

Additional information is available for the following registered author(s):

Abstract

This chapter gives a survey on the use of statistical designs for what-if analysis in simulation, including sensitivity analysis, optimization, and validation/verification. Sensitivity analysis is divided into two phases. The first phase is a pilot stage, which consists of screening or searching for the important factors among (say) hundreds of potentially important factors. A novel screening technique is presented, namely sequential bifurcation. The second phase uses regression analysis to approximate the input/output transformation that is implied by the simulation model; the resulting regression model is also known as a metamodel or a response surface. Regression analysis gives better results when the simu- lation experiment is well designed, using either classical statistical designs (such as fractional factorials) or optimal designs (such as pioneered by Fedorov, Kiefer, and Wolfo- witz). To optimize the simulated system, the analysts may apply Response Surface Methodology (RSM); RSM combines regression analysis, statistical designs, and steepest-ascent hill-climbing. To validate a simulation model, again regression analysis and statistical designs may be applied. Several numerical examples and case-studies illustrate how statisti- cal techniques can reduce the ad hoc character of simulation; that is, these statistical techniques can make simulation studies give more general results, in less time. Appendix 1 summarizes confidence intervals for expected values, proportions, and quantiles, in termi- nating and steady-state simulations. Appendix 2 gives details on four variance reduction techniques, namely common pseudorandom numbers, antithetic numbers, control variates or regression sampling, and importance sampling. Appendix 3 describes jackknifing, which may give robust confidence intervals.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://arno.uvt.nl/show.cgi?fid=3572
File Format: application/pdf
File Function:
Download Restriction: no
File URL: http://arno.uvt.nl/show.cgi?fid=3573
File Format: application/postscript
File Function:
Download Restriction: no

Publisher Info
Paper provided by Tilburg University, Center for Economic Research in its series Discussion Paper with number 52.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length:
Date of creation: 1997
Date of revision:
Handle: RePEc:dgr:kubcen:199752

Contact details of provider:
Web page: http://center.uvt.nl

For technical questions regarding this item, or to correct its listing, contact: (Corry Stuyts).

Related research
Keywords:

Other versions of this item:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
  1. Sherif, Yosef S. & Boice, Bruce A., 1994. "Optimization by pattern search," European Journal of Operational Research, Elsevier, vol. 78(3), pages 277-303, November. [Downloadable!] (restricted)
  2. Kleijnen, Jack P. C. & Standridge, Charles R., 1988. "Experimental design and regression analysis in simulation: An FMS case study," European Journal of Operational Research, Elsevier, vol. 33(3), pages 257-261, February. [Downloadable!] (restricted)
  3. Dagenais, Marcel G & Dufour, Jean-Marie, 1994. "Pitfalls of Rescaling Regression Modes with Box-Cox Transformations," The Review of Economics and Statistics, MIT Press, vol. 76(3), pages 571-75, August. [Downloadable!] (restricted)
    Other versions:
  4. Saltelli, A. & Andres, T. H. & Homma, T., 1995. "Sensitivity analysis of model output. Performance of the iterated fractional factorial design method," Computational Statistics & Data Analysis, Elsevier, vol. 20(4), pages 387-407, October. [Downloadable!] (restricted)
  5. Bettonvil, Bert & Kleijnen, Jack P. C., 1997. "Searching for important factors in simulation models with many factors: Sequential bifurcation," European Journal of Operational Research, Elsevier, vol. 96(1), pages 180-194, January. [Downloadable!] (restricted)
  6. Kleijnen, Jack P. C., 1995. "Verification and validation of simulation models," European Journal of Operational Research, Elsevier, vol. 82(1), pages 145-162, April. [Downloadable!] (restricted)
  7. Kleijnen, Jack P. C., 1995. "Statistical validation of simulation models," European Journal of Operational Research, Elsevier, vol. 87(1), pages 21-34, November. [Downloadable!] (restricted)
  8. Kleijnen, J.P.C., 1995. "Sensitivity Analysis and Optimization of System Dynamics Models : Regression Analysis and Statistical Design of Experiments," Discussion Paper 4, Tilburg University, Center for Economic Research. [Downloadable!]
Full references

Cited by:
(explanations, Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.)

  1. Giorgio Fagiolo & Paul Windrum & Alessio Moneta, 2006. "Empirical Validation of Agent Based Models: A Critical Survey," LEM Papers Series 2006/14, Laboratory of Economics and Management (LEM), Sant'Anna School of Advanced Studies, Pisa, Italy. [Downloadable!]
  2. Matteo Richiardi, 2004. "The Promises and Perils of Agent-Based Computational Economics," Computational Economics 0401001, EconWPA. [Downloadable!]
    Other versions:
  3. H.G. Neddermeijer & G.J. van Oortmarssen & N. Piersma & R. Dekker, 2000. "A framework for response surface methodology for simulation optimization," Econometric Institute Report 192, Erasmus University Rotterdam, Econometric Institute. [Downloadable!]
    Other versions:
  4. Kleijnen, Jack P.C., 2006. "Generalized response surface methodology : a new metaheuristic," Discussion Paper 77, Tilburg University, Center for Economic Research. [Downloadable!]
  5. Kurt DeMaagd & Scott Moore, 2007. "Computational modeling of city formation," Computational Economics, Springer, vol. 30(1), pages 41-56, August. [Downloadable!] (restricted)
  6. Grazia Vicario & Daniele Romano, 2001. "Factorial experiments for sequential process," Metron - International Journal of Statistics, Dipartimento di Statistica, Probabilità e Statistiche Applicate - University of Rome, vol. 0(1-2), pages 191-204. [Downloadable!]
  7. Beers, W.C.M. van & Kleijnen, J.P.C., 2001. "Kriging for interpolation in random simulation," Discussion Paper 74, Tilburg University, Center for Economic Research. [Downloadable!]
  8. Kleijnen, J.P.C., 2001. "Experimental design for sensitivity analysis of simulation models," Discussion Paper 15, Tilburg University, Center for Economic Research. [Downloadable!]
  9. Carlo Bianchi & Pasquale Cirillo & Mauro Gallegati & Pietro Vagliasindi, 2007. "Validating and Calibrating Agent-Based Models: A Case Study," Computational Economics, Springer, vol. 30(3), pages 245-264, October. [Downloadable!] (restricted)
    Other versions:
  10. H.G. Neddermeijer & G.J. van Oortmarssen & N. Piersma & R. Dekker, 2000. "Adaptive extensions of the Nelder and Mead Simplex Method for optimization of stochastic simulation models," Econometric Institute Report 199, Erasmus University Rotterdam, Econometric Institute. [Downloadable!]
    Other versions:
  11. Gaury, E.G.A. & Kleijnen, J.P.C. & Pierreval, H., 1998. "Customized pull systems for single-product flow lines," Discussion Paper 117, Tilburg University, Center for Economic Research. [Downloadable!]
  12. H.G. Neddermeijer & N. Piersma & G.J. van Oortmarssen & J.D.F. Habbema & R. Dekker, 1999. "Comparison of response surface methodology and the Nelder and Mead simplex method for optimization in microsimulation models," Econometric Institute Report 157, Erasmus University Rotterdam, Econometric Institute. [Downloadable!]
    Other versions:
  13. Kleijnen, J.P.C., 2004. "An overview of the design and analysis of simulation experiments for sensitivity analysis," Discussion Paper 16, Tilburg University, Center for Economic Research. [Downloadable!]
    Other versions:
Statistics
Access and download statistics

Did you know? IDEAS also indexes books.

This page was last updated on 2009-10-29.


This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.