Model Validation and Forecast Comparisons : Theoretical and Practical Considerations

Most macroeconometric models are built with the objective, wholly or partly, of providing forecasts. The term "forecast" covers three rather distinct types of exercise : (a) genuine "ex-ante" forecasts, in which the model user predicts the actual future development of the economy, and for which projected future values of input variables must be supplied ; (b) "ex-post" forecasts, in which the model user eliminates the effects of error in the projections of the input variables by calculating "forecasts" over some period in the recent past, given the actual observed values of the input variables ; (c) hypothetical forecasting or policy analysis exercises, in which the model user estimates the response of the economy to the economy to alternative scenarios, that is, to alternative values of policy instruments or to different kinds of exogenous shock. In each case there is interest in evaluating the results of the forecasting exercise, not only for its own sake but also to provide information that is useful in model validation, that is, in checking the specification of the model. Of course the various forecasting exercises and their respective evaluations are not necessarily independent of one another, for example it is often said that in order to be useful in policy analysis a model should have a good real-world forecasting record over a period that was not part of the estimation period, so that it might also be expected to provide "good" estimates of responses to policy changes.