Model Selection in Univariate Time Series Forecasting

A number of studies in the last couple of decades has attempted to find, in terms of postsample accuracy, the best forecasting. procedure for a given set of time series; see Newbold and Granger (1974), Reid (1975), Makridakis and Hibon (1979), Makridakis et al. (1982) and Makridakis et al. (1993). A general conclusion, based on empirical studies, has been that no one forecasting procedure is better than all others for all time series. In other words the name of the game is horses for courses. In this paper, we investigate the possibility of using statistical -discriminant analysis to do forecast model selection. Several techniques, both parametric and nonparametric, are considered. Their performance is compared using four sets of feature vectors and six error rate measures. The results on the quarterly time series of the M-Competition data set show that a number of these techniques are better, and some significantly so, at selecting the most accurate of three forecasting procedures than the within-sample-mean-squared-error criterion. Furthermore, there is strong evidence to suggest that, when an appropriate forecasting procedure is selected using one of these techniques for each time series, then any overall cost is likely to be substantially less than when a single forecasting procedure is selected for all time series.