IDEAS home Printed from https://ideas.repec.org/a/oup/jfinec/v20y2022i3p437-471..html
   My bibliography  Save this article

Regression-Based Expected Shortfall Backtesting
[Backtesting Expected Shortfall]

Author

Listed:
  • Sebastian Bayer
  • Timo Dimitriadis

Abstract

This article introduces novel backtests for the risk measure Expected Shortfall (ES) following the testing idea of Mincer and Zarnowitz (1969). Estimating a regression model for the ES stand-alone is infeasible and thus, our tests are based on a joint regression model for the Value at Risk (VaR) and the ES, which allows for different test specifications. These ES backtests are the first which solely backtest the ES in the sense that they only require ES forecasts as input variables. As the tests are potentially subject to model misspecification, we provide asymptotic theory under misspecification for the underlying joint regression. We find that employing a misspecification robust covariance estimator substantially improves the tests’ performance. We compare our backtests to existing joint VaR and ES backtests and find that our tests outperform the existing alternatives throughout all considered simulations. In an empirical illustration, we apply our backtests to ES forecasts for 200 stocks of the S&P 500 index.

Suggested Citation

  • Sebastian Bayer & Timo Dimitriadis, 2022. "Regression-Based Expected Shortfall Backtesting [Backtesting Expected Shortfall]," Journal of Financial Econometrics, Oxford University Press, vol. 20(3), pages 437-471.
  • Handle: RePEc:oup:jfinec:v:20:y:2022:i:3:p:437-471.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/jjfinec/nbaa013
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Komunjer, Ivana, 2005. "Quasi-maximum likelihood estimation for conditional quantiles," Journal of Econometrics, Elsevier, vol. 128(1), pages 137-164, September.
    2. Natalia Nolde & Johanna F. Ziegel, 2016. "Elicitability and backtesting: Perspectives for banking regulation," Papers 1608.05498, arXiv.org, revised Feb 2017.
    3. Jacob A. Mincer & Victor Zarnowitz, 1969. "The Evaluation of Economic Forecasts," NBER Chapters, in: Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance, pages 3-46, National Bureau of Economic Research, Inc.
    4. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    5. Nelson, Daniel B, 1991. "Conditional Heteroskedasticity in Asset Returns: A New Approach," Econometrica, Econometric Society, vol. 59(2), pages 347-370, March.
    6. Gaglianone, Wagner Piazza & Lima, Luiz Renato & Linton, Oliver & Smith, Daniel R., 2011. "Evaluating Value-at-Risk Models via Quantile Regression," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 150-160.
    7. Johanna F. Ziegel, 2016. "Coherence And Elicitability," Mathematical Finance, Wiley Blackwell, vol. 26(4), pages 901-918, October.
    8. Harvey,Andrew C., 2013. "Dynamic Models for Volatility and Heavy Tails," Cambridge Books, Cambridge University Press, number 9781107630024, January.
    9. Siem Jan Koopman & André Lucas & Marcel Scharth, 2016. "Predicting Time-Varying Parameters with Parameter-Driven and Observation-Driven Models," The Review of Economics and Statistics, MIT Press, vol. 98(1), pages 97-110, March.
    10. White,Halbert, 1996. "Estimation, Inference and Specification Analysis," Cambridge Books, Cambridge University Press, number 9780521574464, January.
    11. Righi, Marcelo Brutti & Ceretta, Paulo Sergio, 2015. "A comparison of Expected Shortfall estimation models," Journal of Economics and Business, Elsevier, vol. 78(C), pages 14-47.
    12. Paul Embrechts & Haiyan Liu & Tiantian Mao & Ruodu Wang, 2017. "Quantile-Based Risk Sharing with Heterogeneous Beliefs," Swiss Finance Institute Research Paper Series 17-65, Swiss Finance Institute, revised Jan 2018.
    13. Patton, Andrew J. & Ziegel, Johanna F. & Chen, Rui, 2019. "Dynamic semiparametric models for expected shortfall (and Value-at-Risk)," Journal of Econometrics, Elsevier, vol. 211(2), pages 388-413.
    14. McNeil, Alexander J. & Frey, Rudiger, 2000. "Estimation of tail-related risk measures for heteroscedastic financial time series: an extreme value approach," Journal of Empirical Finance, Elsevier, vol. 7(3-4), pages 271-300, November.
    15. Jacob A. Mincer, 1969. "Economic Forecasts and Expectations: Analysis of Forecasting Behavior and Performance," NBER Books, National Bureau of Economic Research, Inc, number minc69-1, March.
    16. Rama Cont & Romain Deguest & Giacomo Scandolo, 2010. "Robustness and sensitivity analysis of risk measurement procedures," Post-Print hal-00413729, HAL.
    17. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    18. Glosten, Lawrence R & Jagannathan, Ravi & Runkle, David E, 1993. "On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks," Journal of Finance, American Finance Association, vol. 48(5), pages 1779-1801, December.
    19. White, Halbert, 1980. "Using Least Squares to Approximate Unknown Regression Functions," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 21(1), pages 149-170, February.
    20. Bollerslev, Tim, 1986. "Generalized autoregressive conditional heteroskedasticity," Journal of Econometrics, Elsevier, vol. 31(3), pages 307-327, April.
    21. Bellini, Fabio & Klar, Bernhard & Müller, Alfred & Rosazza Gianin, Emanuela, 2014. "Generalized quantiles as risk measures," Insurance: Mathematics and Economics, Elsevier, vol. 54(C), pages 41-48.
    22. Kemal Guler & Pin T. Ng & Zhijie Xiao, 2017. "Mincer–Zarnowitz quantile and expectile regressions for forecast evaluations under aysmmetric loss functions," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 36(6), pages 651-679, September.
    23. Gourieroux, Christian & Monfort, Alain & Trognon, Alain, 1984. "Pseudo Maximum Likelihood Methods: Theory," Econometrica, Econometric Society, vol. 52(3), pages 681-700, May.
    24. Holden, K & Peel, D A, 1990. "On Testing for Unbiasedness and Efficiency of Forecasts," The Manchester School of Economic & Social Studies, University of Manchester, vol. 58(2), pages 120-127, June.
    25. Gourieroux, Christian & Monfort, Alain & Trognon, Alain, 1984. "Pseudo Maximum Likelihood Methods: Applications to Poisson Models," Econometrica, Econometric Society, vol. 52(3), pages 701-720, May.
    26. Robert F. Engle & Jeffrey R. Russell, 1998. "Autoregressive Conditional Duration: A New Model for Irregularly Spaced Transaction Data," Econometrica, Econometric Society, vol. 66(5), pages 1127-1162, September.
    27. Koenker, Roger W & Bassett, Gilbert, Jr, 1978. "Regression Quantiles," Econometrica, Econometric Society, vol. 46(1), pages 33-50, January.
    28. Weiss, Andrew A., 1991. "Estimating Nonlinear Dynamic Models Using Least Absolute Error Estimation," Econometric Theory, Cambridge University Press, vol. 7(1), pages 46-68, March.
    29. Kerkhof, Jeroen & Melenberg, Bertrand, 2004. "Backtesting for risk-based regulatory capital," Journal of Banking & Finance, Elsevier, vol. 28(8), pages 1845-1865, August.
    30. Rama Cont & Romain Deguest & Giacomo Scandolo, 2010. "Robustness and sensitivity analysis of risk measurement procedures," Quantitative Finance, Taylor & Francis Journals, vol. 10(6), pages 593-606.
    31. Wong, Woon K., 2008. "Backtesting trading risk of commercial banks using expected shortfall," Journal of Banking & Finance, Elsevier, vol. 32(7), pages 1404-1415, July.
    32. Joshua Angrist & Victor Chernozhukov & Iván Fernández-Val, 2006. "Quantile Regression under Misspecification, with an Application to the U.S. Wage Structure," Econometrica, Econometric Society, vol. 74(2), pages 539-563, March.
    33. Nick Costanzino & Michael Curran, 2018. "A Simple Traffic Light Approach to Backtesting Expected Shortfall," Risks, MDPI, vol. 6(1), pages 1-7, January.
    34. Saralees Nadarajah & Bo Zhang & Stephen Chan, 2014. "Estimation methods for expected shortfall," Quantitative Finance, Taylor & Francis Journals, vol. 14(2), pages 271-291, February.
    35. Stefan Weber, 2006. "Distribution‐Invariant Risk Measures, Information, And Dynamic Consistency," Mathematical Finance, Wiley Blackwell, vol. 16(2), pages 419-441, April.
    36. Zaichao Du & Juan Carlos Escanciano, 2017. "Backtesting Expected Shortfall: Accounting for Tail Risk," Management Science, INFORMS, vol. 63(4), pages 940-958, April.
    37. Drew Creal & Siem Jan Koopman & André Lucas, 2013. "Generalized Autoregressive Score Models With Applications," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 28(5), pages 777-795, August.
    38. Yamai, Yasuhiro & Yoshiba, Toshinao, 2002. "On the Validity of Value-at-Risk: Comparative Analyses with Expected Shortfall," Monetary and Economic Studies, Institute for Monetary and Economic Studies, Bank of Japan, vol. 20(1), pages 57-85, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Li, Dan & Clements, Adam & Drovandi, Christopher, 2023. "A Bayesian approach for more reliable tail risk forecasts," Journal of Financial Stability, Elsevier, vol. 64(C).
    2. Hotta, Luiz Koodi & Trucíos Maza, Carlos César & Pereira, Pedro L. Valls & Zevallos Herencia, Mauricio Henrique, 2024. "Forecasting VaR and ES through Markov-switching GARCH models: does the specication matter?," Textos para discussão 567, FGV EESP - Escola de Economia de São Paulo, Fundação Getulio Vargas (Brazil).
    3. Cathy W. S. Chen & Takaaki Koike & Wei-Hsuan Shau, 2024. "Tail risk forecasting with semi-parametric regression models by incorporating overnight information," Papers 2402.07134, arXiv.org.
    4. Hertrich, Daniel, 2023. "Carry and conditional value at risk trend: Capturing the short-, intermediate-, and long-term trends of left-tail risk forecasts," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 82(C).
    5. Ning Zhang & Yujing Gong & Xiaohan Xue, 2023. "Less disagreement, better forecasts: Adjusted risk measures in the energy futures market," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 43(10), pages 1332-1372, October.
    6. Storti, Giuseppe & Wang, Chao, 2022. "A multivariate semi-parametric portfolio risk optimization and forecasting framework," MPRA Paper 115266, University Library of Munich, Germany.
    7. Yannick Hoga & Matei Demetrescu, 2023. "Monitoring Value-at-Risk and Expected Shortfall Forecasts," Management Science, INFORMS, vol. 69(5), pages 2954-2971, May.
    8. Larbi Ait-Hennani & Zoulikha Kaid & Ali Laksaci & Mustapha Rachdi, 2022. "Nonparametric Estimation of the Expected Shortfall Regression for Quasi-Associated Functional Data," Mathematics, MDPI, vol. 10(23), pages 1-23, November.
    9. Jack Fosten & Daniel Gutknecht & Marc-Oliver Pohle, 2023. "Testing Quantile Forecast Optimality," Papers 2302.02747, arXiv.org, revised Oct 2023.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sebastian Bayer & Timo Dimitriadis, 2018. "Regression Based Expected Shortfall Backtesting," Papers 1801.04112, arXiv.org, revised Sep 2019.
    2. Timo Dimitriadis & Xiaochun Liu & Julie Schnaitmann, 2020. "Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary," Papers 2009.07341, arXiv.org.
    3. Dimitriadis, Timo & Schnaitmann, Julie, 2021. "Forecast encompassing tests for the expected shortfall," International Journal of Forecasting, Elsevier, vol. 37(2), pages 604-621.
    4. Timo Dimitriadis & Tobias Fissler & Johanna Ziegel, 2020. "The Efficiency Gap," Papers 2010.14146, arXiv.org, revised Sep 2022.
    5. James Ming Chen, 2018. "On Exactitude in Financial Regulation: Value-at-Risk, Expected Shortfall, and Expectiles," Risks, MDPI, vol. 6(2), pages 1-28, June.
    6. Patton, Andrew J. & Ziegel, Johanna F. & Chen, Rui, 2019. "Dynamic semiparametric models for expected shortfall (and Value-at-Risk)," Journal of Econometrics, Elsevier, vol. 211(2), pages 388-413.
    7. David Happersberger & Harald Lohre & Ingmar Nolte, 2020. "Estimating portfolio risk for tail risk protection strategies," European Financial Management, European Financial Management Association, vol. 26(4), pages 1107-1146, September.
    8. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    9. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.
    10. Gerlach, Richard & Wang, Chao, 2020. "Semi-parametric dynamic asymmetric Laplace models for tail risk forecasting, incorporating realized measures," International Journal of Forecasting, Elsevier, vol. 36(2), pages 489-506.
    11. Santos, Douglas G. & Candido, Osvaldo & Tófoli, Paula V., 2022. "Forecasting risk measures using intraday and overnight information," The North American Journal of Economics and Finance, Elsevier, vol. 60(C).
    12. Sander Barendse & Erik Kole & Dick van Dijk, 2023. "Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 528-568.
    13. Blasques, Francisco & van Brummelen, Janneke & Koopman, Siem Jan & Lucas, André, 2022. "Maximum likelihood estimation for score-driven models," Journal of Econometrics, Elsevier, vol. 227(2), pages 325-346.
    14. Santiago Carrillo Menéndez & Bertrand Kian Hassani, 2021. "Expected Shortfall Reliability—Added Value of Traditional Statistics and Advanced Artificial Intelligence for Market Risk Measurement Purposes," Mathematics, MDPI, vol. 9(17), pages 1-20, September.
    15. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    16. Ophélie Couperier & Jérémy Leymarie, 2020. "Backtesting Expected Shortfall via Multi-Quantile Regression," Working Papers halshs-01909375, HAL.
    17. Timo Dimitriadis & Julie Schnaitmann, 2019. "Forecast Encompassing Tests for the Expected Shortfall," Papers 1908.04569, arXiv.org, revised Aug 2020.
    18. Astrid Ayala & Szabolcs Blazsek, 2018. "Equity market neutral hedge funds and the stock market: an application of score-driven copula models," Applied Economics, Taylor & Francis Journals, vol. 50(37), pages 4005-4023, August.
    19. Ruodu Wang & Ričardas Zitikis, 2021. "An Axiomatic Foundation for the Expected Shortfall," Management Science, INFORMS, vol. 67(3), pages 1413-1429, March.
    20. Enrique Molina‐Muñoz & Andrés Mora‐Valencia & Javier Perote, 2021. "Backtesting expected shortfall for world stock index ETFs with extreme value theory and Gram–Charlier mixtures," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 4163-4189, July.

    More about this item

    Keywords

    asymptotic theory; backtesting; expected shortfall; forecast evaluation; Mincer–Zarnowitz regression; model misspecification;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics
    • G32 - Financial Economics - - Corporate Finance and Governance - - - Financing Policy; Financial Risk and Risk Management; Capital and Ownership Structure; Value of Firms; Goodwill

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:jfinec:v:20:y:2022:i:3:p:437-471.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://edirc.repec.org/data/sofieea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.