Testing Whether Volatility Model Gains Persist: A Prespecified Holdout in Tail Risk Forecasting

Rankings of volatility models often change with the market, the evaluation period, and the forecast criterion, so a gain found in one setting may not carry over to another. We test this directly. Eight volatility models, four observation-driven and four stochastic volatility, are estimated for five equity indices by data cloning and compared on one-step-ahead forecasts. We assess joint value-at-risk and expected shortfall accuracy, calibration, predictive density accuracy, and forecast availability. In 2018–2023, nominally significant gains cluster in DAX and NIKKEI, with weaker evidence in FTSE. Neither holdout comparison is corroborated: the DAX estimate changes sign, and the NIKKEI estimate keeps its direction but is imprecise. Different criteria also favour different models. The results argue for validating a model in its intended market and for the criterion it is meant to serve, and they show how a prespecified holdout in a later period can test whether an advantage persists.