Backtesting macroprudential stress tests

Macroprudential stress tests generate a wide range of stress outcomes, depending on the chosen input parameters. Building on the concept of reverse stress tests, we embrace this parameter sensitivity in a backtesting exercise. We generalize an otherwise standard model of price-mediated contagion by interpolating between different liquidation dynamics among banks (leverage targeting vs. threshold dynamics). We then test the capability of this model to match actual bank non-/defaults in the United States for the years 2008–10, where we treat the underlying liquidation dynamics as another free input parameter. While the model performance depends on the type of shock being imposed, we find that all liquidation dynamics we consider can explain to some extent (in particular better than a random benchmark) the pattern of defaults observed during the subprime crisis. We identify the region in the parameter space where a specific dynamic leads to the best fit of the data, and in the most relevant regime (illiquid asset markets and small initial shocks) leverage targeting turns out to provide the most accurate results. We also show how the results depend on the initial shock level, the market impact parameter, on the number of asset liquidation rounds, and the chosen liquidation functions.