This file is part of IDEAS, which uses RePEc data


[ Papers | Articles | Software | Books | Chapters | Authors | Institutions | JEL Classification | NEP reports | Search | New papers by email | Author registration | Rankings | Volunteers | FAQ | Blog | Help! ]

Model Evaluation Using Stochastic Simulations: The Case of the Econometric Model KOSMOS

Author info | Abstract | Publisher info | Download info | Related research | Statistics
Author Info
Gajda, Jan B. (National Institute of Economic Research)
Markowski, Aleksander () (National Institute of Economic Research)
Abstract

One aspect of model behaviour that is of interest to the model builder is sensitivity to different forms of errors. This can be investigated using stochastic simulations, as shown by Gajda [1995]. The method involves generating random numbers from a given (usually normal) distribution and introducing them as shocks to the model. In particular, stochastic simulations can be employed to make an empirical investigation into the following aspects of the model:

The results of stochastic simulations can provide information on - inter alia - the sampling distribution of the model forecast. In particular, the model builder may be interested in the shape of this distribution. If it is not symmetric, the mean (stochastic) forecast will be different from the median forecast (which under certain conditions is equal to the deterministic forecast). Furthermore, if the distribution is skewed, a typical stochastic forecast (represented by the mode) will systematically underestimate (or overestimate) both the mean and the median forecasts.

The purpose of the present paper is to investigate KOSMOS, the econometric model of the National Institute of Economic Research in Stockholm, from the point of view of the first three aspects mentioned above. The main aim of this exercise is to look for ”weak links” in the model, i.e. to find out which equations introduce most uncertainty and at the same time are crucial for the forecast because of their strong influence on it. Thus, our interest is not only in assessing the forecast error variance as a descriptive statistic, but also - and primarily - in finding those equations that are important for error propagation and those coefficients whose values are crucial for the model.

The outline of the paper is as follows. Section 2 discusses the analysis of expected forecast errors (for linear models) based on analytical formulae. In Section 3, model simulations and stochastic simulations are defined. The two subsequent sections discuss the purpose of our experiments and their design, respectively. Section 6 gives a brief description of the econometric model KOSMOS, whose equations are subject to our investigation. Section 7 describes the results of stochastic simulations with additive equation disturbances. Section 8 presents the results of stochastic simulations with both equation disturbances and disturbances to the estimated coefficients. Sampling distributions of forecasts are discussed in Section 9. Section 10 concludes.

Download Info
To download:

If you experience problems downloading a file, check if you have the proper application to view it first. Information about this may be contained in the File-Format links below. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

File URL: http://www.konj.se/download/18.2f48d2f18732142c7fff533/Wp61.pdf
File Format: application/pdf
File Function:
Download Restriction: no

Publisher Info
Paper provided by National Institute of Economic Research in its series Working Paper with number 61.

Download reference. The following formats are available: HTML (with abstract), plain text (with abstract), BibTeX, RIS (EndNote, RefMan, ProCite), ReDIF
Length: 41 pages
Date of creation: 01 Aug 1998
Date of revision:
Handle: RePEc:hhs:nierwp:0061

Contact details of provider:
Postal: National Institute of Economic Research, P.O. Box 3116, SE-103 62 Stockholm, Sweden
Phone: 46-(0)8-453 59 00
Fax: 46-(0)8-453 59 80
Email:
Web page: http://www.konj.se/
More information through EDIRC

For technical questions regarding this item, or to correct its listing, contact: (Britta Larsson).

Related research
Keywords:

References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

  1. Mariano, Roberto S, 1982. "Analytical Small-Sample Distribution Theory in Econometrics: The Simultaneous-Equations Case," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 23(3), pages 503-33, October. [Downloadable!] (restricted)
  2. Hall, S G, 1986. "Estimating the Uncertainty of the Simulation Properties of Large Nonlinear Econometric Models," Applied Economics, Taylor and Francis Journals, vol. 18(9), pages 985-93, September.
  3. Calzolari, Giorgio, 1979. "Antithetic variates to estimate the simulation bias in non-linear models," Economics Letters, Elsevier, vol. 4(4), pages 323-328. [Downloadable!] (restricted)
  4. Feldstein, Martin S, 1971. "The Error of Forecast in Econometric Models when the Forecast-Period Exogenous Variables are Stochastic," Econometrica, Econometric Society, vol. 39(1), pages 55-60, January.
  5. Brown, Bryan W & Mariano, Roberto S, 1984. "Residual-Based Procedures for Prediction and Estimation in a Nonlinear Simultaneous System," Econometrica, Econometric Society, vol. 52(2), pages 321-43, March. [Downloadable!] (restricted)
  6. Hendry, David F., 1984. "Monte carlo experimentation in econometrics," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 16, pages 937-976 Elsevier. [Downloadable!] (restricted)
  7. Fisher, Paul & Salmon, Mark, 1986. "On Evaluating the Importance of Nonlinearity in Large Macroeconometric Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 27(3), pages 625-46, October. [Downloadable!] (restricted)
    Other versions:
  8. Brown, Bryan W & Mariano, Roberto S, 1989. "Measures of Deterministic Prediction Bias in Nonlinear Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 30(3), pages 667-84, August. [Downloadable!] (restricted)
  9. Bianchi, Carlo & Calzolari, Giorgio & Corsi, Paolo, 1979. "A Monte Carlo approach to compute the asymptotic standard errors of dynamic multipliers," Economics Letters, Elsevier, vol. 2(2), pages 161-164. [Downloadable!] (restricted)
  10. Bianchi, Carlo & Calzolari, Giorgio, 1980. "The One-Period Forecast Errors in Nonlinear Econometric Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(1), pages 201-08, February. [Downloadable!] (restricted)
  11. Fair, Ray C & Taylor, John B, 1983. "Solution and Maximum Likelihood Estimation of Dynamic Nonlinear Rational Expectations Models," Econometrica, Econometric Society, vol. 51(4), pages 1169-85, July. [Downloadable!] (restricted)
    Other versions:
  12. Wallis, Kenneth F., 1982. "'Time-series' versus 'econometric' forecasts : A non-linear regression counterexample," Economics Letters, Elsevier, vol. 10(3-4), pages 309-315. [Downloadable!] (restricted)
  13. Hall, S G, 1986. "The Application of Stochastic Simulation Techniques to the National Institute's Model 7," The Manchester School of Economic & Social Studies, Blackwell Publishing, vol. 54(2), pages 180-201, June.
  14. Salmon, Mark & Wallis, Kenneth F, 1980. "Model Validation and Forecast Comparisons : Theoretical and Practical Considerations," The Warwick Economics Research Paper Series (TWERPS) 184, University of Warwick, Department of Economics.
  15. Bianchi, Carlo & Calzolari, Giorgio & Corsi, Paolo, 1981. "Estimating asymptotic standard errors and inconsistencies of impact multipliers in nonlinear econometric models," Journal of Econometrics, Elsevier, vol. 16(3), pages 277-294, August. [Downloadable!] (restricted)
  16. Bianchi, Carlo & Calzolari, Giorgio & Brillet, Jean-Louis, 1987. "Measuring forecast uncertainty : A review with evaluation based on a macro model of the French economy," International Journal of Forecasting, Elsevier, vol. 3(2), pages 211-227. [Downloadable!] (restricted)
  17. Mariano, Roberto S & Brown, Bryan W, 1983. "Asymptotic Behavior of Predictors in a Nonlinear Simultaneous System," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 24(3), pages 523-36, October. [Downloadable!] (restricted)
  18. Fair, Ray C, 1980. "Estimating the Uncertainty of Policy Effects in Nonlinear Models," Econometrica, Econometric Society, vol. 48(6), pages 1381-91, September. [Downloadable!] (restricted)
  19. Sterbenz, Frederic P & Calzolari, Giorgio, 1990. "Alternative Specifications of the Error Process in the Stochastic Simulation of Econometric Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 5(2), pages 137-50, April-Jun. [Downloadable!] (restricted)
  20. Corker, R. J. & Holly, S. & Ellis, R. G., 1986. "Uncertainty and forecast precision," International Journal of Forecasting, Elsevier, vol. 2(1), pages 53-53. [Downloadable!] (restricted)
Full references

Statistics
Access and download statistics

Did you know? IDEAS also computes impact factors for journals and working paper series.

This page was last updated on 2009-11-18.


This information is provided to you by IDEAS at the Department of Economics, College of Liberal Arts and Sciences, University of Connecticut using RePEc data on a server sponsored by the Society for Economic Dynamics.