Improved Tests for Forecast Comparisons in the Presence of Instabilities

Of interest is comparing the out-of-sample forecasting performance of two competing models in the presence of possible instabilities. To that effect, we suggest using simple structural change tests, sup-Wald and UDmax as proposed by Andrews (1993) and Bai and Perron (1998), for changes in the mean of the loss-differences. Giacomini and Rossi (2010) proposed a áuctuations test and a one-time reversal test also applied to the loss-differences. When properly constructed to account for potential serial correlation under the null hypothesis to have a pivotal limit distribution, it is shown that their tests have undesirable power properties, power that can be low and non-increasing as the alternative gets further from the null hypothesis. The good power properties they reported is simply an artifact of imposing a priori that the loss di§erentials are serially uncorrelated and using the simple sample variance to scale the tests. On the contrary, our statistics are shown to have higher monotonic power, especially the UDmax version. We use their empirical examples to show the practical relevance of the issues raised.