IDEAS home Printed from https://ideas.repec.org/a/spr/empeco/v61y2021i3d10.1007_s00181-020-01905-4.html
   My bibliography  Save this article

Backtesting and estimation error: value-at-risk overviolation rate

Author

Listed:
  • Georges Tsafack

    (University of Rhode Island)

  • James Cataldo

    (Citizens Financial Group, Model Validation)

Abstract

Financial institutions and regulators use value-at-risk (VaR) and related measures as a tool for financial risk management. It is therefore critical to appropriately assess the quality of VaR forecasts and reporting. The VaR estimation error creates an additional source of imprecision. We show that even an unbiased estimator of VaR is likely to produce a systematic overviolation. We then propose an adjustment to account for the issue. A Monte Carlo study illustrates the overviolation problem and the effectiveness of the adjustment. An application to Fama–French portfolios returns series highlights the need to further account for tail behavior in the data. Applying the adjustment to the normal distribution performs relatively well for a less prudential level (5% VaR), but is unable to provide enough buffer to overcome the overviolation for more prudential levels (1% or 0.5%VaR). Using the empirical distribution for more prudential levels improves risk forecasts.

Suggested Citation

  • Georges Tsafack & James Cataldo, 2021. "Backtesting and estimation error: value-at-risk overviolation rate," Empirical Economics, Springer, vol. 61(3), pages 1351-1396, September.
  • Handle: RePEc:spr:empeco:v:61:y:2021:i:3:d:10.1007_s00181-020-01905-4
    DOI: 10.1007/s00181-020-01905-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00181-020-01905-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00181-020-01905-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Kerkhof, Jeroen & Melenberg, Bertrand & Schumacher, Hans, 2010. "Model risk and capital reserves," Journal of Banking & Finance, Elsevier, vol. 34(1), pages 267-279, January.
    2. Wen Cheong Chin & Min Cherng Lee, 2018. "S&P500 volatility analysis using high-frequency multipower variation volatility proxies," Empirical Economics, Springer, vol. 54(3), pages 1297-1318, May.
    3. John Y. Campbell & Luis M. Viceira, 1999. "Consumption and Portfolio Decisions when Expected Returns are Time Varying," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 433-495.
    4. Koenker, Roger & Xiao, Zhijie, 2006. "Quantile Autoregression," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 980-990, September.
    5. Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value‐at‐Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, June.
    6. Andrew Ang & Geert Bekaert, 2002. "International Asset Allocation With Regime Shifts," The Review of Financial Studies, Society for Financial Studies, vol. 15(4), pages 1137-1187.
    7. Tauchen, George & Hussey, Robert, 1991. "Quadrature-Based Methods for Obtaining Approximate Solutions to Nonlinear Asset Pricing Models," Econometrica, Econometric Society, vol. 59(2), pages 371-396, March.
    8. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
    9. Christoffersen, Peter, 2011. "Elements of Financial Risk Management," Elsevier Monographs, Elsevier, edition 2, number 9780123744487.
    10. Gourieroux, C. & Jasiak, J., 2008. "Dynamic quantile models," Journal of Econometrics, Elsevier, vol. 147(1), pages 198-205, November.
    11. Loffler, Gunter, 2003. "The effects of estimation error on measures of portfolio credit risk," Journal of Banking & Finance, Elsevier, vol. 27(8), pages 1427-1453, August.
    12. Carol Alexander & José María Sarabia, 2012. "Quantile Uncertainty and Value‐at‐Risk Model Risk," Risk Analysis, John Wiley & Sons, vol. 32(8), pages 1293-1308, August.
    13. Pérignon, Christophe & Deng, Zi Yin & Wang, Zhi Jun, 2008. "Do banks overstate their Value-at-Risk?," Journal of Banking & Finance, Elsevier, vol. 32(5), pages 783-794, May.
    14. Şener, Emrah & Baronyan, Sayad & Ali Mengütürk, Levent, 2012. "Ranking the predictive performances of value-at-risk estimation methods," International Journal of Forecasting, Elsevier, vol. 28(4), pages 849-873.
    15. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    16. Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011. "Evaluating Value-at-Risk Models with Desk-Level Data," Management Science, INFORMS, vol. 57(12), pages 2213-2227, December.
    17. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    18. Balduzzi, Pierluigi & Lynch, Anthony W., 1999. "Transaction costs and predictability: some utility cost calculations," Journal of Financial Economics, Elsevier, vol. 52(1), pages 47-78, April.
    19. A. Amendola & V. Candila, 2016. "Evaluation of volatility predictions in a VaR framework," Quantitative Finance, Taylor & Francis Journals, vol. 16(5), pages 695-709, May.
    20. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Oxford University Press, vol. 2(1), pages 84-108.
    21. Zaichao Du & Juan Carlos Escanciano, 2017. "Backtesting Expected Shortfall: Accounting for Tail Risk," Management Science, INFORMS, vol. 63(4), pages 940-958, April.
    22. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    23. Felix Scheller & Benjamin R. Auer, 2018. "How does the choice of Value-at-Risk estimator influence asset allocation decisions?," Quantitative Finance, Taylor & Francis Journals, vol. 18(12), pages 2005-2022, December.
    24. Leccadito, Arturo & Boffelli, Simona & Urga, Giovanni, 2014. "Evaluating the accuracy of value-at-risk forecasts: New multilevel tests," International Journal of Forecasting, Elsevier, vol. 30(2), pages 206-216.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    2. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    3. Argyropoulos, Christos & Panopoulou, Ekaterini, 2019. "Backtesting VaR and ES under the magnifying glass," International Review of Financial Analysis, Elsevier, vol. 64(C), pages 22-37.
    4. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    5. Gordy, Michael B. & McNeil, Alexander J., 2020. "Spectral backtests of forecast distributions with application to risk management," Journal of Banking & Finance, Elsevier, vol. 116(C).
    6. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    7. Denisa Banulescu-Radu & Christophe Hurlin & Jérémy Leymarie & Olivier Scaillet, 2021. "Backtesting Marginal Expected Shortfall and Related Systemic Risk Measures," Management Science, INFORMS, vol. 67(9), pages 5730-5754, September.
    8. Thiele, Stephen, 2019. "Detecting underestimates of risk in VaR models," Journal of Banking & Finance, Elsevier, vol. 101(C), pages 12-20.
    9. Farkas, Walter & Fringuellotti, Fulvia & Tunaru, Radu, 2020. "A cost-benefit analysis of capital requirements adjusted for model risk," Journal of Corporate Finance, Elsevier, vol. 65(C).
    10. Komunjer, Ivana, 2013. "Quantile Prediction," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 961-994, Elsevier.
    11. James M. O'Brien & Pawel J. Szerszen, 2014. "An Evaluation of Bank VaR Measures for Market Risk During and Before the Financial Crisis," Finance and Economics Discussion Series 2014-21, Board of Governors of the Federal Reserve System (U.S.).
    12. Liu, Wei & Semeyutin, Artur & Lau, Chi Keung Marco & Gozgor, Giray, 2020. "Forecasting Value-at-Risk of Cryptocurrencies with RiskMetrics type models," Research in International Business and Finance, Elsevier, vol. 54(C).
    13. Colletaz, Gilbert & Hurlin, Christophe & Pérignon, Christophe, 2013. "The Risk Map: A new tool for validating risk models," Journal of Banking & Finance, Elsevier, vol. 37(10), pages 3843-3854.
    14. O’Brien, James & Szerszeń, Paweł J., 2017. "An evaluation of bank measures for market risk before, during and after the financial crisis," Journal of Banking & Finance, Elsevier, vol. 80(C), pages 215-234.
    15. Benjamin R. Auer & Benjamin Mögel, 2016. "How Accurate are Modern Value-at-Risk Estimators Derived from Extreme Value Theory?," CESifo Working Paper Series 6288, CESifo.
    16. Escanciano, Juan Carlos & Pei, Pei, 2012. "Pitfalls in backtesting Historical Simulation VaR models," Journal of Banking & Finance, Elsevier, vol. 36(8), pages 2233-2244.
    17. Frésard, Laurent & Pérignon, Christophe & Wilhelmsson, Anders, 2011. "The pernicious effects of contaminated data in risk management," Journal of Banking & Finance, Elsevier, vol. 35(10), pages 2569-2583, October.
    18. Benjamin Mögel & Benjamin R. Auer, 2018. "How accurate are modern Value-at-Risk estimators derived from extreme value theory?," Review of Quantitative Finance and Accounting, Springer, vol. 50(4), pages 979-1030, May.
    19. Radu Tunaru, 2015. "Model Risk in Financial Markets:From Financial Engineering to Risk Management," World Scientific Books, World Scientific Publishing Co. Pte. Ltd., number 9524, January.
    20. Gaglianone, Wagner Piazza & Marins, Jaqueline Terra Moura, 2017. "Evaluation of exchange rate point and density forecasts: An application to Brazil," International Journal of Forecasting, Elsevier, vol. 33(3), pages 707-728.

    More about this item

    Keywords

    Risk management; Value-at-risk; Forecasting; Backtesting; Estimation error;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:empeco:v:61:y:2021:i:3:d:10.1007_s00181-020-01905-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.