Validation of Models: Statistical Techniques and Data Availability
AbstractThis paper shows which statistical techniques can be used to validate simulation models, depending on which real-life data are available. Concerning this availability three situations are distinguished (i) no data, (ii) only output data, and (iii) both input and output data. In case (i) - no real data - the analysts can still experiment with the simulation model to obtain simulated data; such an experiment should be guided by the statistical theory on the design of experiments. In case (ii) - only output data - real and simulated output data can be compared through the well-known two-sample Student t statistic or certain other statistics. In case (iii) - input and output data - trace-driven simulation becomes possible, but validation should not proceed in the popular way (make a scatter plot with real and simulated outputs, fit a line, and test whether that line has unit slope and passes through the origin); alternative regression and bootstrap procedures are presented. Several case studies are summarized, to illustrate the three types of situations.
Download InfoIf you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
Bibliographic InfoPaper provided by Tilburg University, Center for Economic Research in its series Discussion Paper with number 1999-104.
Date of creation: 1999
Date of revision:
Contact details of provider:
Web page: http://center.uvt.nl
Statistical methods; simulation models;
Other versions of this item:
- Kleijnen, J.P.C., 1999. "Validation of models: Statistical techniques and data availability," Open Access publications from Tilburg University urn:nbn:nl:ui:12-81791, Tilburg University.
- NEP-ALL-1999-11-28 (All new papers)
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Kleijnen, J.P.C. & Pala, O., 1999. "Maximizing the simulation output: A competition," Open Access publications from Tilburg University urn:nbn:nl:ui:12-80658, Tilburg University.
- Robinson, Stewart, 2002. "General concepts of quality for discrete-event simulation," European Journal of Operational Research, Elsevier, vol. 138(1), pages 103-117, April.
- Kleijnen, J.P.C., 2001.
"Ethical issues in modeling: Some reflections,"
Open Access publications from Tilburg University
urn:nbn:nl:ui:12-85771, Tilburg University.
- Federico Perali & Stefania Lovo, 2009. "Counterfactual analysis using a regional dynamic general equilibrium model with historical calibration," Working Papers 58/2009, University of Verona, Department of Economics.
- Hertel, Thomas W. & Keeney, Roman & Valenzuela, Ernesto, 2004. "Global Analysis Of Agricultural Trade Liberalization: Assessing Model Validity," 2004 Annual meeting, August 1-4, Denver, CO 20199, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
- van der Gaag, Monique A. & Vos, Fred & Saatkamp, Helmut W. & van Boven, Michiel & van Beek, Paul & Huirne, Ruud B. M., 2004. "A state-transition simulation model for the spread of Salmonella in the pork supply chain," European Journal of Operational Research, Elsevier, vol. 156(3), pages 782-798, August.
- Happe, Kathrin, 2005. "Agent-Based Modelling and Sensitivity Analysis by Experimental Design and Metamodelling: An Application to Modelling Regional Structural Change," 2005 International Congress, August 23-27, 2005, Copenhagen, Denmark 24464, European Association of Agricultural Economists.
- Kleijnen, Jack P.C., 2011. "Ethical issues in engineering models: An operations researcher's reflections," Open Access publications from Tilburg University urn:nbn:nl:ui:12-4274966, Tilburg University.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Richard Broekman).
If references are entirely missing, you can add them using this form.