Comparing Forecast Performance with State Dependence

We propose a novel forecast comparison methodology to evaluate models’ relative forecasting performance when the latter is a state-dependent function of economic variables. In our bench¬mark case, the relative forecasting performance, measured by the forecast loss differential, is modeled via a threshold model. Importantly, we allow the threshold that triggers the switch from one state to the next to be unknown, leading to a non-standard test statistic due to the presence of a nuisance parameter. Existing tests either assume a constant out-of-sample forecast performance or use non-parametric techniques robust to time-variation; consequently, they may lack power against state-dependent predictability. Importantly, our approach is applicable to point forecasts as well as predictive densities. Monte Carlo results suggest that our proposed test statistics perform well in ï¬ nite samples and have better power than existing tests in selecting the best forecasting model in the presence of state dependence. Our test statistics uncover “pockets of predictability†in U.S. equity premia forecasts; the pockets are a state-dependent function of stock market volatility. Models using economic predictors perform signiï¬ cantly worse than a simple mean forecast in periods of high volatility, but, in periods of low volatility, the use of economic predictors may lead to small forecast improvements.