Assessing the accuracy of directional forecasts

Directional predictions are used in a variety of settings, including economics, finance, meteorology and medicine. Since the objectives in each setting can be quite different, appropriate tests for assessing the predictions are necessary. After summarizing existing tests, we propose new tests to assess the accuracy for cases where getting extreme events wrong is particularly costly and where there is serial dependence in small samples. We show in simulations that the latter test has a more favourable power-size trade-off relative to existing tests. The new tests are then applied to the directional EUR/USD forecasts in the ifo-Institute’s World Economic Survey and to the point forecasts in the Philadelphia Fed’s Survey of Professional Forecasters. For the former, the new tests show that while there is profitability and predictive accuracy in some cases, a simple time-series model is not beaten. For the latter, extreme values are not the main driver of the predictive value up to two quarters ahead. This shows that the additional tests allow a more precise determination of predictive accuracy and economic value and where it is coming from.