IDEAS home Printed from https://ideas.repec.org/a/eee/jbfina/v101y2019icp12-20.html
   My bibliography  Save this article

Detecting underestimates of risk in VaR models

Author

Listed:
  • Thiele, Stephen

Abstract

This paper explores avenues to improve VaR backtesting against violations of unconditional coverage, with emphasis on detecting underestimates of risk. I show that existing tests for unconditional coverage have an undesirable asymmetry in power, making them less (more) effective at detecting underestimates (overestimates) of risk. A set of Lagrange multiplier (LM) tests is proposed, leading to improved power against the more dangerous underestimates. Also, the use of one-sided coverage tests is revisited, with a one-sided version of the new LM test performing best. Results are supported by simulations, showing the effectiveness of the new tests in finite samples. The same issues of asymmetric power are shown to carry over to the case of conditional coverage, though it appears that finite sample issues affecting the independence component of the tests can adversely impact performance. A simple weighted test statistic is shown to help alleviate this problem, providing an avenue to exploit the advantages of the LM approach in a conditional coverage setting.

Suggested Citation

  • Thiele, Stephen, 2019. "Detecting underestimates of risk in VaR models," Journal of Banking & Finance, Elsevier, vol. 101(C), pages 12-20.
    • Handle: RePEc:eee:jbfina:v:101:y:2019:i:c:p:12-20
    DOI: 10.1016/j.jbankfin.2019.01.018
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0378426619300251
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jbankfin.2019.01.018?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Bertrand Candelon & Gilbert Colletaz & Christophe Hurlin & Sessi Tokpavi, 2011. "Backtesting Value-at-Risk: A GMM Duration-Based Test," Journal of Financial Econometrics, Oxford University Press, vol. 9(2), pages 314-343, Spring.
    2. Gaglianone, Wagner Piazza & Lima, Luiz Renato & Linton, Oliver & Smith, Daniel R., 2011. "Evaluating Value-at-Risk Models via Quantile Regression," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 150-160.
    3. Dufour, Jean-Marie, 2006. "Monte Carlo tests with nuisance parameters: A general approach to finite-sample inference and nonstandard asymptotics," Journal of Econometrics, Elsevier, vol. 133(2), pages 443-477, August.
    4. Demos, Antonis & Sentana, Enrique, 1998. "Testing for GARCH effects: a one-sided approach," Journal of Econometrics, Elsevier, vol. 86(1), pages 97-127, June.
    5. Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011. "Evaluating Value-at-Risk Models with Desk-Level Data," Management Science, INFORMS, vol. 57(12), pages 2213-2227, December.
    6. Davidson, Russell & MacKinnon, James G., 1984. "Convenient specification tests for logit and probit models," Journal of Econometrics, Elsevier, vol. 25(3), pages 241-262, July.
    7. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    8. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    9. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    10. Giacomini, Raffaella & Komunjer, Ivana, 2005. "Evaluation and Combination of Conditional Quantile Forecasts," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 416-431, October.
    11. Denis Pelletier & Wei Wei, 2016. "The Geometric-VaR Backtesting Method," Journal of Financial Econometrics, Oxford University Press, vol. 14(4), pages 725-745.
    12. Pena D. & Rodriguez J., 2002. "A Powerful Portmanteau Test of Lack of Fit for Time Series," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 601-610, June.
    13. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    14. Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
    15. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    16. Wong, Woon K., 2010. "Backtesting value-at-risk based on tail losses," Journal of Empirical Finance, Elsevier, vol. 17(3), pages 526-538, June.
    17. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    18. Esam Mahdi & A. Ian McLeod, 2012. "Improved multivariate portmanteau test," Journal of Time Series Analysis, Wiley Blackwell, vol. 33(2), pages 211-222, March.
    19. Ziggel, Daniel & Berens, Tobias & Weiß, Gregor N.F. & Wied, Dominik, 2014. "A new set of improved Value-at-Risk backtests," Journal of Banking & Finance, Elsevier, vol. 48(C), pages 29-41.
    20. Andrew J. Patton, 2006. "Modelling Asymmetric Exchange Rate Dependence," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 47(2), pages 527-556, May.
    21. Gourieroux, Christian & Holly, Alberto & Monfort, Alain, 1982. "Likelihood Ratio Test, Wald Test, and Kuhn-Tucker Test in Linear Models with Inequality Constraints on the Regression Parameters," Econometrica, Econometric Society, vol. 50(1), pages 63-80, January.
    22. Zaichao Du & Juan Carlos Escanciano, 2017. "Backtesting Expected Shortfall: Accounting for Tail Risk," Management Science, INFORMS, vol. 63(4), pages 940-958, April.
    23. J. Carlos Escanciano & Jose Olmo, 2011. "Robust Backtesting Tests for Value-at-risk Models," Journal of Financial Econometrics, Oxford University Press, vol. 9(1), pages 132-161, Winter.
    24. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    2. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    3. Sullivan Hu'e & Christophe Hurlin & Yang Lu, 2024. "Backtesting Expected Shortfall: Accounting for both duration and severity with bivariate orthogonal polynomials," Papers 2405.02012, arXiv.org, revised May 2024.
    4. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    5. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    6. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    7. Yannick Hoga & Matei Demetrescu, 2023. "Monitoring Value-at-Risk and Expected Shortfall Forecasts," Management Science, INFORMS, vol. 69(5), pages 2954-2971, May.
    8. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.
    9. Jean-Paul Laurent & Hassan Omidi Firouzi, 2022. "Market Risk and Volatility Weighted Historical Simulation After Basel III," Working Papers hal-03679434, HAL.
    10. Denisa Banulescu-Radu & Christophe Hurlin & Jérémy Leymarie & Olivier Scaillet, 2021. "Backtesting Marginal Expected Shortfall and Related Systemic Risk Measures," Management Science, INFORMS, vol. 67(9), pages 5730-5754, September.
    11. Timo Dimitriadis & Xiaochun Liu & Julie Schnaitmann, 2020. "Encompassing Tests for Value at Risk and Expected Shortfall Multi-Step Forecasts based on Inference on the Boundary," Papers 2009.07341, arXiv.org.
    12. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    13. Sander Barendse & Erik Kole & Dick van Dijk, 2023. "Backtesting Value-at-Risk and Expected Shortfall in the Presence of Estimation Error," Journal of Financial Econometrics, Oxford University Press, vol. 21(2), pages 528-568.
    14. Elena-Ivona Dumitrescu & Christophe Hurlin & Vinson Pham, 2012. "Backtesting Value-at-Risk: From Dynamic Quantile to Dynamic Binary Tests," Finance, Presses universitaires de Grenoble, vol. 33(1), pages 79-112.
    15. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    16. Wied, Dominik & Weiß, Gregor N.F. & Ziggel, Daniel, 2016. "Evaluating Value-at-Risk forecasts: A new set of multivariate backtests," Journal of Banking & Finance, Elsevier, vol. 72(C), pages 121-132.
    17. Marta Małecka & Radosław Pietrzyk, 2024. "A spectral approach to evaluating VaR forecasts: stock market evidence from the subprime mortgage crisis, through COVID-19, to the Russo–Ukrainian war," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(5), pages 4533-4567, October.
    18. Soren Bettels & Sojung Kim & Stefan Weber, 2022. "Multinomial Backtesting of Distortion Risk Measures," Papers 2201.06319, arXiv.org, revised Aug 2024.
    19. Marta Małecka, 2021. "Testing for a serial correlation in VaR failures through the exponential autoregressive conditional duration model," Statistics in Transition New Series, Polish Statistical Association, vol. 22(1), pages 145-162, March.
    20. Kratz, Marie & Lok, Yen H. & McNeil, Alexander J., 2018. "Multinomial VaR backtests: A simple implicit approach to backtesting expected shortfall," Journal of Banking & Finance, Elsevier, vol. 88(C), pages 393-407.

    More about this item

    Keywords

    Risk management; Model validation; One-sided alternatives; Lagrange multiplier tests;
    All these keywords.

    JEL classification:

    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • C58 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Financial Econometrics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jbfina:v:101:y:2019:i:c:p:12-20. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jbf .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.