IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2007.12431.html
   My bibliography  Save this paper

Tile test for back-testing risk evaluation

Author

Listed:
  • Gilles Zumbach

Abstract

A new test for measuring the accuracy of financial market risk estimations is introduced. It is based on the probability integral transform (PIT) of the ex post realized returns using the ex ante probability distributions underlying the risk estimation. If the forecast is correct, the result of the PIT, that we called probtile, should be an iid random variable with a uniform distribution. The new test measures the variance of the number of probtiles in a tiling over the whole sample. Using different tilings allow to check the dynamic and the distributional aspect of risk methodologies. The new test is very powerful, and new benchmarks need to be introduced to take into account subtle mean reversion effects induced by some risk estimations. The test is applied on 2 data sets for risk horizons of 1 and 10 days. The results show unambiguously the importance of capturing correctly the dynamic of the financial market, and exclude some broadly used risk methodologies.

Suggested Citation

  • Gilles Zumbach, 2020. "Tile test for back-testing risk evaluation," Papers 2007.12431, arXiv.org.
  • Handle: RePEc:arx:papers:2007.12431
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2007.12431
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fajardo, J. & Farias, A. R & Ornelas, J. R. H, 2003. "Goodness-of-fit Tests focus on VaR Estimation," Finance Lab Working Papers flwp_55, Finance Lab, Insper Instituto de Ensino e Pesquisa.
    2. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    3. Gilles Zumbach, 2004. "Volatility processes and volatility forecast with long memory," Quantitative Finance, Taylor & Francis Journals, vol. 4(1), pages 70-86.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bauwens, Luc & Giot, Pierre & Grammig, Joachim & Veredas, David, 2004. "A comparison of financial duration models via density forecasts," International Journal of Forecasting, Elsevier, vol. 20(4), pages 589-609.
    2. Hendry, David F. & Clements, Michael P., 2003. "Economic forecasting: some lessons from recent research," Economic Modelling, Elsevier, vol. 20(2), pages 301-329, March.
    3. Ivanova, Vesela & Puigvert Gutiérrez, Josep Maria, 2014. "Interest rate forecasts, state price densities and risk premium from Euribor options," Journal of Banking & Finance, Elsevier, vol. 48(C), pages 210-223.
    4. Li, Xiao-Ming & Rose, Lawrence C., 2009. "The tail risk of emerging stock markets," Emerging Markets Review, Elsevier, vol. 10(4), pages 242-256, December.
    5. Franses, Ph.H.B.F. & Kranendonk, H.C. & Lanser, D., 2007. "On the optimality of expert-adjusted forecasts," Econometric Institute Research Papers EI 2007-38, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    6. Maik H. Wolters, 2015. "Evaluating Point and Density Forecasts of DSGE Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 30(1), pages 74-96, January.
    7. González-Rivera, Gloria & Sun, Yingying, 2017. "Density forecast evaluation in unstable environments," International Journal of Forecasting, Elsevier, vol. 33(2), pages 416-432.
    8. David Kohns & Tibor Szendrei, 2021. "Decoupling Shrinkage and Selection for the Bayesian Quantile Regression," Papers 2107.08498, arXiv.org.
    9. Andrea Carriero & Todd E. Clark & Massimiliano Marcellino, 2015. "Realtime nowcasting with a Bayesian mixed frequency model with stochastic volatility," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 837-862, October.
    10. M. Hashem Pesaran & Paolo Zaffaroni, 2004. "Model Averaging and Value-at-Risk Based Evaluation of Large Multi Asset Volatility Models for Risk Management," CESifo Working Paper Series 1358, CESifo.
    11. Helton Saulo & Jeremias Leão & Víctor Leiva & Robert G. Aykroyd, 2019. "Birnbaum–Saunders autoregressive conditional duration models applied to high-frequency financial data," Statistical Papers, Springer, vol. 60(5), pages 1605-1629, October.
    12. Yun, Jaeho, 2014. "Out-of-sample density forecasts with affine jump diffusion models," Journal of Banking & Finance, Elsevier, vol. 47(C), pages 74-87.
    13. Sentana, Enrique & Calzolari, Giorgio & Fiorentini, Gabriele, 2008. "Indirect estimation of large conditionally heteroskedastic factor models, with an application to the Dow 30 stocks," Journal of Econometrics, Elsevier, vol. 146(1), pages 10-25, September.
    14. Diebold, Francis X. & Li, Canlin, 2006. "Forecasting the term structure of government bond yields," Journal of Econometrics, Elsevier, vol. 130(2), pages 337-364, February.
    15. Jian Wang & Jason J. Wu, 2012. "The Taylor Rule and Forecast Intervals for Exchange Rates," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(1), pages 103-144, February.
    16. Wolfgang Polasek, 2013. "Forecast Evaluations for Multiple Time Series: A Generalized Theil Decomposition," Working Paper series 23_13, Rimini Centre for Economic Analysis.
    17. Monica Billio & Roberto Casarin & Francesco Ravazzolo & Herman K. van Dijk, 2010. "Combining predictive densities using Bayesian filtering with applications to US economics data," Working Paper 2010/29, Norges Bank.
    18. Mauro Bernardi & Leopoldo Catania, 2015. "Switching-GAS Copula Models With Application to Systemic Risk," Papers 1504.03733, arXiv.org, revised Jan 2016.
    19. Corradi, Valentina & Swanson, Norman R., 2004. "A test for the distributional comparison of simulated and historical data," Economics Letters, Elsevier, vol. 85(2), pages 185-193, November.
    20. Vasileios Siakoulis & Ioannis Venetis, 2015. "On inter-arrival times of bond market extreme events. An application to seven European markets," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 39(4), pages 717-741, October.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2007.12431. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.