Advanced Search
MyIDEAS: Login to save this paper or follow this series

Model Evaluation Using Stochastic Simulations: The Case of the Econometric Model KOSMOS

Contents:

Author Info

  • Gajda, Jan B.

    (National Institute of Economic Research)

  • Markowski, Aleksander

    ()
    (National Institute of Economic Research)

Registered author(s):

    Abstract

    One aspect of model behaviour that is of interest to the model builder is sensitivity to different forms of errors. This can be investigated using stochastic simulations, as shown by Gajda [1995]. The method involves generating random numbers from a given (usually normal) distribution and introducing them as shocks to the model. In particular, stochastic simulations can be employed to make an empirical investigation into the following aspects of the model: the effects on the forecast of random disturbances, the effects on the forecast of random variation in equation parameters (sampling errors), error propagation and accumulation patterns in the model, the effects on the forecast of (random) errors in the exogenous variables. The results of stochastic simulations can provide information on - inter alia - the sampling distribution of the model forecast. In particular, the model builder may be interested in the shape of this distribution. If it is not symmetric, the mean (stochastic) forecast will be different from the median forecast (which under certain conditions is equal to the deterministic forecast). Furthermore, if the distribution is skewed, a typical stochastic forecast (represented by the mode) will systematically underestimate (or overestimate) both the mean and the median forecasts. The purpose of the present paper is to investigate KOSMOS, the econometric model of the National Institute of Economic Research in Stockholm, from the point of view of the first three aspects mentioned above. The main aim of this exercise is to look for ”weak links” in the model, i.e. to find out which equations introduce most uncertainty and at the same time are crucial for the forecast because of their strong influence on it. Thus, our interest is not only in assessing the forecast error variance as a descriptive statistic, but also - and primarily - in finding those equations that are important for error propagation and those coefficients whose values are crucial for the model. The outline of the paper is as follows. Section 2 discusses the analysis of expected forecast errors (for linear models) based on analytical formulae. In Section 3, model simulations and stochastic simulations are defined. The two subsequent sections discuss the purpose of our experiments and their design, respectively. Section 6 gives a brief description of the econometric model KOSMOS, whose equations are subject to our investigation. Section 7 describes the results of stochastic simulations with additive equation disturbances. Section 8 presents the results of stochastic simulations with both equation disturbances and disturbances to the estimated coefficients. Sampling distributions of forecasts are discussed in Section 9. Section 10 concludes.

    Download Info

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
    File URL: http://www.konj.se/download/18.70c52033121865b1398800093414/WP_61.pdf
    Download Restriction: no

    Bibliographic Info

    Paper provided by National Institute of Economic Research in its series Working Paper with number 61.

    as in new window
    Length: 41 pages
    Date of creation: 01 Aug 1998
    Date of revision:
    Handle: RePEc:hhs:nierwp:0061

    Contact details of provider:
    Postal: National Institute of Economic Research, P.O. Box 3116, SE-103 62 Stockholm, Sweden
    Phone: 46-(0)8-453 59 00
    Fax: 46-(0)8-453 59 80
    Email:
    Web page: http://www.konj.se/
    More information through EDIRC

    Related research

    Keywords:

    References

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
    as in new window
    1. Bianchi, Carlo & Calzolari, Giorgio & Corsi, Paolo, 1981. "Standard errors of multipliers and forecasts from structural coefficients with block-diagonal covariance matrix," MPRA Paper 22678, University Library of Munich, Germany, revised 1981.
    2. Bianchi, Carlo & Calzolari, Giorgio & Corsi, Paolo, 1979. "A Monte Carlo approach to compute the asymptotic standard errors of dynamic multipliers," Economics Letters, Elsevier, vol. 2(2), pages 161-164.
    3. Mariano, Roberto S & Brown, Bryan W, 1983. "Asymptotic Behavior of Predictors in a Nonlinear Simultaneous System," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 24(3), pages 523-36, October.
    4. Oliver, F. R., 1979. "Rebased and revised data -- their effects on econometric estimates," Economics Letters, Elsevier, vol. 4(4), pages 355-359.
    5. Sterbenz, Frederic P & Calzolari, Giorgio, 1990. "Alternative Specifications of the Error Process in the Stochastic Simulation of Econometric Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 5(2), pages 137-50, April-Jun.
    6. Fair, Ray C, 1980. "Estimating the Uncertainty of Policy Effects in Nonlinear Models," Econometrica, Econometric Society, vol. 48(6), pages 1381-91, September.
    7. Hall, S G, 1986. "The Application of Stochastic Simulation Techniques to the National Institute's Model 7," The Manchester School of Economic & Social Studies, University of Manchester, vol. 54(2), pages 180-201, June.
    8. Hendry, David F., 1984. "Monte carlo experimentation in econometrics," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 16, pages 937-976 Elsevier.
    9. Ray C. Fair, 1978. "Estimating the Expected Predictive Accuracy of Econometric Models," Cowles Foundation Discussion Papers 480, Cowles Foundation for Research in Economics, Yale University.
    10. Bianchi, Carlo & Calzolari, Giorgio & Corsi, Paolo, 1976. "Divergences in the results of stochastic and deterministic simulation of an Italian non linear econometric model," MPRA Paper 21287, University Library of Munich, Germany.
    11. Wallis, Kenneth F., 1982. "'Time-series' versus 'econometric' forecasts : A non-linear regression counterexample," Economics Letters, Elsevier, vol. 10(3-4), pages 309-315.
    12. Brown, Bryan W & Mariano, Roberto S, 1984. "Residual-Based Procedures for Prediction and Estimation in a Nonlinear Simultaneous System," Econometrica, Econometric Society, vol. 52(2), pages 321-43, March.
    13. Bianchi, Carlo & Calzolari, Giorgio & Corsi, Paolo, 1981. "Estimating asymptotic standard errors and inconsistencies of impact multipliers in nonlinear econometric models," Journal of Econometrics, Elsevier, vol. 16(3), pages 277-294, August.
    14. Ray C. Fair & John B. Taylor, 1980. "Solution and Maximum Likelihood Estimation of Dynamic Nonlinear RationalExpectations Models," NBER Technical Working Papers 0005, National Bureau of Economic Research, Inc.
    15. Calzolari, Giorgio, 1979. "Antithetic variates to estimate the simulation bias in non-linear models," Economics Letters, Elsevier, vol. 4(4), pages 323-328.
    16. Brillet, Jean-Louis & Calzolari, Giorgio & Panattoni, Lorenzo, 1986. "Coherent optimal prediction with large nonlinear systems: an example based on a French model," MPRA Paper 29057, University Library of Munich, Germany.
    17. Brown, Bryan W & Mariano, Roberto S, 1989. "Measures of Deterministic Prediction Bias in Nonlinear Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 30(3), pages 667-84, August.
    18. Mariano, Roberto S, 1982. "Analytical Small-Sample Distribution Theory in Econometrics: The Simultaneous-Equations Case," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 23(3), pages 503-33, October.
    19. Salmon, Mark & Wallis, Kenneth F, 1980. "Model Validation and Forecast Comparisons : Theoretical and Practical Considerations," The Warwick Economics Research Paper Series (TWERPS) 184, University of Warwick, Department of Economics.
    20. Bianchi, Carlo & Calzolari, Giorgio & Weihs, Claus, 1986. "Parametric and nonparametric Monte Carlo estimates of standard errors of forecasts in econometric models," MPRA Paper 29120, University Library of Munich, Germany.
    21. repec:sae:niesru:v:145:y::i:1:p:43-63 is not listed on IDEAS
    22. Bianchi, Carlo & Calzolari, Giorgio, 1982. "Evaluating forecast uncertainty due to errors in estimated coefficients: empirical comparison of alternative methods," MPRA Paper 22559, University Library of Munich, Germany.
    23. Bianchi, Carlo & Calzolari, Giorgio, 1980. "The One-Period Forecast Errors in Nonlinear Econometric Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(1), pages 201-08, February.
    24. Corker, R. J. & Holly, S. & Ellis, R. G., 1986. "Uncertainty and forecast precision," International Journal of Forecasting, Elsevier, vol. 2(1), pages 53-53.
    25. Fisher, Paul & Salmon, Mark, 1985. "On Evaluating the Importance of Non-Linearity in Large Macroeconometric Models," CEPR Discussion Papers 86, C.E.P.R. Discussion Papers.
    26. Brown, Bryan W. & Mariano, Roberto S., 1989. "Predictors in Dynamic Nonlinear Models: Large-Sample Behavior," Econometric Theory, Cambridge University Press, vol. 5(03), pages 430-452, December.
    27. Bianchi, Carlo & Calzolari, Giorgio & Brillet, Jean-Louis, 1987. "Measuring forecast uncertainty : A review with evaluation based on a macro model of the French economy," International Journal of Forecasting, Elsevier, vol. 3(2), pages 211-227.
    28. Feldstein, Martin S, 1971. "The Error of Forecast in Econometric Models when the Forecast-Period Exogenous Variables are Stochastic," Econometrica, Econometric Society, vol. 39(1), pages 55-60, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Lists

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    Statistics

    Access and download statistics

    Corrections

    When requesting a correction, please mention this item's handle: RePEc:hhs:nierwp:0061. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Henrik Hellström).

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.