In this paper we systematically compare forecasting accuracy of hypothesis testing procedures with that of a model combining algorithm. Testing procedures are commonly used in applications to select a model, based on which forecasts are made. However, besides the well-known difficulty in dealing with multiple tests, the testing approach has a potentially serious drawback: controlling the probability of Type I error at a conventional level (e.g., 0.05) often excessively favors the null, which can be problematic for the purpose of forecasting. In addition, as shown in this paper, testing procedures can be very unstable, which results in high variability in the forecasts.Selecting a candidate forecast by testing and combining forecasts are both useful but for complementary situations. Currently, there seems to be little guidance in the literature on when combining should be preferred to selecting. We propose instability measures that are helpful for a forecaster to gauge the difficulty in selecting a single optimal forecast.Based on empirical evidences and theoretical considerations, we advocate the use of forecast combining when there is considerable instability in model selection by testing procedures. On the other hand, when there is little instability, testing procedures could work well or even better than forecast combining in terms of forecast accuracy.
Download Info
To download:
If you experience problems downloading a file, check if you have the
proper application to
view it first. Information about this may be contained
in the File-Format links below. In case of further problems read
the IDEAS help
file. Note that these files are not on the IDEAS
site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
References listed on IDEAS Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
Graham Elliott & Allan Timmermann, 2005.
"Optimal Forecast Combination Under Regime Switching ,"
International Economic Review,
Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 46(4), pages 1081-1102, November.
[Downloadable!] (restricted)
Other versions:
David F. Hendry & Michael P. Clements, 2004.
"Pooling of forecasts,"
Econometrics Journal,
Royal Economic Society, vol. 7(1), pages 1-31, 06.
[Downloadable!] (restricted)
Other versions:
David Hendry & Michael P. Clements, 2001.
"Pooling of Forecasts,"
Economics Papers
2002-W9, Economics Group, Nuffield College, University of Oxford.
[Downloadable!]