IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2007.12431.html

Tile test for back-testing risk evaluation

Author

Listed:
  • Gilles Zumbach

Abstract

A new test for measuring the accuracy of financial market risk estimations is introduced. It is based on the probability integral transform (PIT) of the ex post realized returns using the ex ante probability distributions underlying the risk estimation. If the forecast is correct, the result of the PIT, that we called probtile, should be an iid random variable with a uniform distribution. The new test measures the variance of the number of probtiles in a tiling over the whole sample. Using different tilings allow to check the dynamic and the distributional aspect of risk methodologies. The new test is very powerful, and new benchmarks need to be introduced to take into account subtle mean reversion effects induced by some risk estimations. The test is applied on 2 data sets for risk horizons of 1 and 10 days. The results show unambiguously the importance of capturing correctly the dynamic of the financial market, and exclude some broadly used risk methodologies.

Suggested Citation

  • Gilles Zumbach, 2020. "Tile test for back-testing risk evaluation," Papers 2007.12431, arXiv.org.
    • Handle: RePEc:arx:papers:2007.12431
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2007.12431
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fajardo, J. & Farias, A. R & Ornelas, J. R. H, 2003. "Goodness-of-fit Tests focus on VaR Estimation," Finance Lab Working Papers flwp_55, Finance Lab, Insper Instituto de Ensino e Pesquisa.
    2. Diebold, Francis X & Gunther, Todd A & Tay, Anthony S, 1998. "Evaluating Density Forecasts with Applications to Financial Risk Management," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 863-883, November.
    3. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    4. Gilles Zumbach, 2004. "Volatility processes and volatility forecast with long memory," Quantitative Finance, Taylor & Francis Journals, vol. 4(1), pages 70-86.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hendry, David F. & Clements, Michael P., 2003. "Economic forecasting: some lessons from recent research," Economic Modelling, Elsevier, vol. 20(2), pages 301-329, March.
    2. M. Hashem Pesaran & Paolo Zaffaroni, 2004. "Model Averaging and Value-at-Risk Based Evaluation of Large Multi Asset Volatility Models for Risk Management," CESifo Working Paper Series 1358, CESifo.
    3. Mehmet Balcilar & Rangan Gupta & Stephen M. Miller, 2015. "The out-of-sample forecasting performance of nonlinear models of regional housing prices in the US," Applied Economics, Taylor & Francis Journals, vol. 47(22), pages 2259-2277, May.
    4. Allen, David & Lazarov, Zdravetz & McAleer, Michael & Peiris, Shelton, 2009. "Comparison of alternative ACD models via density and interval forecasts: Evidence from the Australian stock market," Mathematics and Computers in Simulation (MATCOM), Elsevier, vol. 79(8), pages 2535-2555.
    5. Corradi, Valentina & Swanson, Norman R., 2004. "Some recent developments in predictive accuracy testing with nested models and (generic) nonlinear alternatives," International Journal of Forecasting, Elsevier, vol. 20(2), pages 185-199.
    6. Marc Gronwald & Janina Ketterer & Stefan Trück, 2011. "The Dependence Structure between Carbon Emission Allowances and Financial Markets - A Copula Analysis," CESifo Working Paper Series 3418, CESifo.
    7. Francisco Peñaranda, 2004. "Are Vector Autoregressions an Accurate Model for Dynamic Asset Allocation?," Working Papers wp2004_0419, CEMFI.
    8. Xun Lu & Kin Lai & Liang Liang, 2014. "Portfolio value-at-risk estimation in energy futures markets with time-varying copula-GARCH model," Annals of Operations Research, Springer, vol. 219(1), pages 333-357, August.
    9. Jose A. Lopez, 1999. "Methods for evaluating value-at-risk estimates," Economic Review, Federal Reserve Bank of San Francisco, pages 3-17.
    10. Warne, Anders & Coenen, Günter & Christoffel, Kai, 2010. "Forecasting with DSGE models," Working Paper Series 1185, European Central Bank.
    11. Refet Gürkaynak & Justin Wolfers, 2005. "Macroeconomic Derivatives: An Initial Analysis of Market-Based Macro Forecasts, Uncertainty, and Risk," NBER Chapters, in: NBER International Seminar on Macroeconomics 2005, pages 11-50, National Bureau of Economic Research, Inc.
    12. Degiannakis, Stavros & Floros, Christos, 2013. "Modeling CAC40 volatility using ultra-high frequency data," Research in International Business and Finance, Elsevier, vol. 28(C), pages 68-81.
    13. Jian Wang & Jason J. Wu, 2012. "The Taylor Rule and Forecast Intervals for Exchange Rates," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 44(1), pages 103-144, February.
    14. Deschamps, Philippe J., 2012. "Bayesian estimation of generalized hyperbolic skewed student GARCH models," Computational Statistics & Data Analysis, Elsevier, vol. 56(11), pages 3035-3054.
    15. Li, Yushu & Andersson, Jonas, 2014. "A Likelihood Ratio and Markov Chain Based Method to Evaluate Density Forecasting," Discussion Papers 2014/12, Norwegian School of Economics, Department of Business and Management Science.
    16. Yushu Li & Jonas Andersson, 2020. "A likelihood ratio and Markov chain‐based method to evaluate density forecasting," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(1), pages 47-55, January.
    17. Kim, Kun Ho, 2011. "Density forecasting through disaggregation," International Journal of Forecasting, Elsevier, vol. 27(2), pages 394-412.
    18. Deschamps, Philippe J., 2011. "Bayesian estimation of an extended local scale stochastic volatility model," Journal of Econometrics, Elsevier, vol. 162(2), pages 369-382, June.
    19. Inoue, Atsushi & Kilian, Lutz, 2006. "On the selection of forecasting models," Journal of Econometrics, Elsevier, vol. 130(2), pages 273-306, February.
    20. David Ardia & Lukasz Gatarek & Lennart F. hoogerheide, 2014. "A New Bootstrap Test for the Validity of a Set of Marginal Models for Multiple Dependent Time Series: an Application to Risk Analysis," Cahiers de recherche 1413, CIRPEE.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2007.12431. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.