IDEAS home Printed from https://ideas.repec.org/p/hal/wpaper/halshs-01909375.html
   My bibliography  Save this paper

Backtesting Expected Shortfall via Multi-Quantile Regression

Author

Listed:
  • Ophélie Couperier

    (CREST - Centre de Recherche en Economie et Statistique [Bruz] - ENSAI - Ecole Nationale de la Statistique et de l'Analyse de l'Information [Bruz])

  • Jérémy Leymarie

    (Faculty of Business, Economics and Statistics - University of Vienna [Vienna])

Abstract

This article proposes an original approach to backtest Expected Shortfall (ES) exploiting the definition of ES as a function of Value-at-Risk (VaR). Our methodology examines jointly the validity of the VaR forecasts along the tail distribution of the risk model, and encompasses the Basel Committee recommendation of verifying quantiles at risk levels 97.5%, and 99%. We introduce four easy-to-use backtests in which we regress the ex-post losses on the VaR forecasts in a multi-quantile regression model, and test the resulting parameter estimates. Monte-Carlo simulations show that our tests are powerful to detect various model misspecifications. We apply our backtests on S&P500 returns over the period 2007-2012. Our tests identify misleading ES forecasts in this period of financial turmoil. Empirical results also show that the detection abilities are higher when the evaluation procedure involves more than two quantiles, which should accordingly be taken into account in the current regulatory guidelines.

Suggested Citation

  • Ophélie Couperier & Jérémy Leymarie, 2020. "Backtesting Expected Shortfall via Multi-Quantile Regression," Working Papers halshs-01909375, HAL.
  • Handle: RePEc:hal:wpaper:halshs-01909375
    Note: View the original document on HAL open archive server: https://shs.hal.science/halshs-01909375v5
    as

    Download full text from publisher

    File URL: https://shs.hal.science/halshs-01909375v5/document
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Christophe Hurlin & Sébastien Laurent & Rogier Quaedvlieg & Stephan Smeekes, 2017. "Risk Measure Inference," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(4), pages 499-512, October.
    2. Gaglianone, Wagner Piazza & Lima, Luiz Renato & Linton, Oliver & Smith, Daniel R., 2011. "Evaluating Value-at-Risk Models via Quantile Regression," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 150-160.
    3. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    4. Danielsson, Jon & Zhou, Chen, 2015. "Why risk is so hard to measure," LSE Research Online Documents on Economics 62002, London School of Economics and Political Science, LSE Library.
    5. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    6. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    7. Powell, James L., 1984. "Least absolute deviations estimation for the censored regression model," Journal of Econometrics, Elsevier, vol. 25(3), pages 303-325, July.
    8. Juan Carlos Escanciano & Zaichao Du, 2015. "Backtesting Expected Shortfall: Accounting for Tail Risk," CAEPR Working Papers 2015-001, Center for Applied Economics and Policy Research, Department of Economics, Indiana University Bloomington.
    9. Acerbi, Carlo & Tasche, Dirk, 2002. "On the coherence of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1487-1503, July.
    10. Christoffersen, Peter, 2011. "Elements of Financial Risk Management," Elsevier Monographs, Elsevier, edition 2, number 9780123744487.
    11. White, Halbert & Kim, Tae-Hwan & Manganelli, Simone, 2015. "VAR for VaR: Measuring tail dependence using multivariate regression quantiles," Journal of Econometrics, Elsevier, vol. 187(1), pages 169-188.
    12. Susanne Emmer & Marie Kratz & Dirk Tasche, 2013. "What is the best risk measure in practice? A comparison of standard measures," Papers 1312.1645, arXiv.org, revised Apr 2015.
    13. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    14. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    15. Manganelli, Simone & White, Halbert & Kim, Tae-Hwan, 2008. "Modeling autoregressive conditional skewness and kurtosis with multi-quantile CAViaR," Working Paper Series 957, European Central Bank.
    16. Sebastian Bayer & Timo Dimitriadis, 2018. "Regression Based Expected Shortfall Backtesting," Papers 1801.04112, arXiv.org, revised Sep 2019.
    17. McNeil, Alexander J. & Frey, Rudiger, 2000. "Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 271-300, November.
    18. Gourieroux, Christian & Zakoïan, Jean-Michel, 2013. "Estimation-Adjusted Var," Econometric Theory, Cambridge University Press, vol. 29(4), pages 735-770, August.
    19. Kerkhof, Jeroen & Melenberg, Bertrand, 2004. "Backtesting for risk-based regulatory capital," Journal of Banking & Finance, Elsevier, vol. 28(8), pages 1845-1865, August.
    20. Wong, Woon K., 2008. "Backtesting trading risk of commercial banks using expected shortfall," Journal of Banking & Finance, Elsevier, vol. 32(7), pages 1404-1415, July.
    21. Nick Costanzino & Michael Curran, 2018. "A Simple Traffic Light Approach to Backtesting Expected Shortfall," Risks, MDPI, vol. 6(1), pages 1-7, January.
    22. Zaichao Du & Juan Carlos Escanciano, 2017. "Backtesting Expected Shortfall: Accounting for Tail Risk," Management Science, INFORMS, vol. 63(4), pages 940-958, April.
    23. Roger Koenker & Zhijie Xiao, 2002. "Inference on the Quantile Regression Process," Econometrica, Econometric Society, vol. 70(4), pages 1583-1612, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Vica Tendenan & Richard Gerlach & Chao Wang, 2020. "Tail risk forecasting using Bayesian realized EGARCH models," Papers 2008.05147, arXiv.org, revised Aug 2020.
    2. Lyócsa, Štefan & Todorova, Neda & Výrost, Tomáš, 2021. "Predicting risk in energy markets: Low-frequency data still matter," Applied Energy, Elsevier, vol. 282(PA).
    3. Marcel, Bräutigam & Marie, Kratz, 2018. "On the Dependence between Quantiles and Dispersion Estimators," ESSEC Working Papers WP1807, ESSEC Research Center, ESSEC Business School.
    4. Timo Dimitriadis & Tobias Fissler & Johanna Ziegel, 2020. "The Efficiency Gap," Papers 2010.14146, arXiv.org, revised Sep 2022.
    5. Dimitriadis, Timo & Liu, Xiaochun & Schnaitmann, Julie, 2020. "Encompassing tests for value at risk and expected shortfall multi-step forecasts based on inference on the boundary," Hohenheim Discussion Papers in Business, Economics and Social Sciences 11-2020, University of Hohenheim, Faculty of Business, Economics and Social Sciences.
    6. Owusu Junior, Peterson & Tiwari, Aviral Kumar & Tweneboah, George & Asafo-Adjei, Emmanuel, 2022. "GAS and GARCH based value-at-risk modeling of precious metals," Resources Policy, Elsevier, vol. 75(C).
    7. Marcel Bräutigam & Marie Kratz, 2018. "On the Dependence between Quantiles and Dispersion Estimators," Working Papers hal-02296832, HAL.
    8. Loïc Maréchal, 2021. "Do economic variables forecast commodity futures volatility?," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 41(11), pages 1735-1774, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    2. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    3. Sebastian Bayer & Timo Dimitriadis, 2018. "Regression Based Expected Shortfall Backtesting," Papers 1801.04112, arXiv.org, revised Sep 2019.
    4. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    5. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    6. Sander Barendse & Erik Kole & Dick van Dijk, 2023. "Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 528-568.
    7. Enrique Molina‐Muñoz & Andrés Mora‐Valencia & Javier Perote, 2021. "Backtesting expected shortfall for world stock index ETFs with extreme value theory and Gram–Charlier mixtures," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 4163-4189, July.
    8. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    9. Timo Dimitriadis & Xiaochun Liu & Julie Schnaitmann, 2020. "Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary," Papers 2009.07341, arXiv.org.
    10. Sebastian Bayer & Timo Dimitriadis, 2022. "Regression-Based Expected Shortfall Backtesting [Backtesting Expected Shortfall]," Journal of Financial Econometrics, Oxford University Press, vol. 20(3), pages 437-471.
    11. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    12. Alfonso Novales & Laura Garcia-Jorcano, 2019. "Backtesting Extreme Value Theory models of expected shortfall," Documentos de Trabajo del ICAE 2019-24, Universidad Complutense de Madrid, Facultad de Ciencias Económicas y Empresariales, Instituto Complutense de Análisis Económico.
    13. Merlo, Luca & Petrella, Lea & Raponi, Valentina, 2021. "Forecasting VaR and ES using a joint quantile regression and its implications in portfolio allocation," Journal of Banking & Finance, Elsevier, vol. 133(C).
    14. Julia S. Mehlitz & Benjamin R. Auer, 2021. "Time‐varying dynamics of expected shortfall in commodity futures markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 41(6), pages 895-925, June.
    15. Marie Kratz & Yen H Lok & Alexander J Mcneil, 2016. "Multinomial var backtests: A simple implicit approach to backtesting expected shortfall," Working Papers hal-01424279, HAL.
    16. Pradhan, Ashis Kumar & Tiwari, Aviral Kumar, 2021. "Estimating the market risk of clean energy technologies companies using the expected shortfall approach," Renewable Energy, Elsevier, vol. 177(C), pages 95-100.
    17. Luca Merlo & Lea Petrella & Valentina Raponi, 2021. "Forecasting VaR and ES using a joint quantile regression and implications in portfolio allocation," Papers 2106.06518, arXiv.org.
    18. Laura Garcia-Jorcano & Lidia Sanchis-Marco, 2023. "Measuring Systemic Risk Using Multivariate Quantile-Located ES Models," Journal of Financial Econometrics, Oxford University Press, vol. 21(1), pages 1-72.
    19. Taylor, James W., 2022. "Forecasting Value at Risk and expected shortfall using a model with a dynamic omega ratio," Journal of Banking & Finance, Elsevier, vol. 140(C).
    20. Kratz, Marie & Lok, Y-H & McNeil, Alexander J., 2016. "Multinomial VaR Backtests: A simple implicit approach to backtesting expected shortfall," ESSEC Working Papers WP1617, ESSEC Research Center, ESSEC Business School.

    More about this item

    Keywords

    Banking regulation; Financial risk management; Forecast evaluation; Hypothesis testing; Tail risk;
    All these keywords.

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:wpaper:halshs-01909375. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.