IDEAS home Printed from https://ideas.repec.org/p/inu/caeprp/2012-003.html
   My bibliography  Save this paper

Pitfalls in Backtesting Historical Simulation VaR Models

Author

Listed:
  • Juan Carlos Escanciano

    () (Indiana University)

  • Pei Pei

    () (Indiana University and Chinese Academy of Finance and Development, Central University of Finance and Economics)

Abstract

Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most widely used Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our ndings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical ndings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. Finally, our theoretical ndings are con rmed in a Monte Carlo simulation study and an empirical application with three U.S. stocks. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.

Suggested Citation

  • Juan Carlos Escanciano & Pei Pei, 2012. "Pitfalls in Backtesting Historical Simulation VaR Models," Caepr Working Papers 2012-003, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
  • Handle: RePEc:inu:caeprp:2012-003
    as

    Download full text from publisher

    File URL: http://www.iub.edu/~caepr/RePEc/PDF/2012/CAEPR2012-003.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Hansen, Bruce E, 1994. "Autoregressive Conditional Density Estimation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 35(3), pages 705-730, August.
    2. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, number 8355, October.
    3. Koenker, Roger & Xiao, Zhijie, 2006. "Quantile Autoregression," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 980-990, September.
    4. Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value-at-Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, June.
    5. Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011. "Evaluating Value-at-Risk Models with Desk-Level Data," Management Science, INFORMS, vol. 57(12), pages 2213-2227, December.
    6. Sun, Shuxia, 2006. "The Bahadur representation for sample quantiles under weak dependence," Statistics & Probability Letters, Elsevier, vol. 76(12), pages 1238-1244, July.
    7. Sean D. Campbell, 2005. "A review of backtesting and backtesting procedures," Finance and Economics Discussion Series 2005-21, Board of Governors of the Federal Reserve System (US).
    8. Peter Christoffersen, 2004. "Backtesting Value-at-Risk: A Duration-Based Approach," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 2(1), pages 84-108.
    9. Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
    10. Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, Elsevier.
    11. Li, W K & Ling, Shiqing & McAleer, Michael, 2002. " Recent Theoretical Results for Time Series Models with GARCH Errors," Journal of Economic Surveys, Wiley Blackwell, vol. 16(3), pages 245-269, July.
    12. Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
    13. Christophe Perignon & D. Smith, 2009. "The Level and Quality of Value-at-Risk Disclosure by Commercial Banks," Post-Print hal-00496102, HAL.
    14. Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (US).
    15. Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
    16. M.J.B. Hall, 1996. "The amendment to the capital accord to incorporate market risk," BNL Quarterly Review, Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
    17. Pritsker, Matthew, 2006. "The hidden dangers of historical simulation," Journal of Banking & Finance, Elsevier, vol. 30(2), pages 561-582, February.
    18. Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
    19. Pérignon, Christophe & Smith, Daniel R., 2010. "The level and quality of Value-at-Risk disclosure by commercial banks," Journal of Banking & Finance, Elsevier, vol. 34(2), pages 362-377, February.
    20. Yoshihara, Ken-ichi, 1995. "The Bahadur representation of sample quantiles for sequences of strongly mixing random variables," Statistics & Probability Letters, Elsevier, vol. 24(4), pages 299-304, September.
    21. Gourieroux, C. & Jasiak, J., 2008. "Dynamic quantile models," Journal of Econometrics, Elsevier, vol. 147(1), pages 198-205, November.
    22. Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(01), pages 17-39, February.
    23. Delgado, Miguel A. & Carlos Escanciano, J., 2007. "Nonparametric tests for conditional symmetry in dynamic models," Journal of Econometrics, Elsevier, vol. 141(2), pages 652-682, December.
    24. J. Carlos Escanciano & Jose Olmo, 2011. "Robust Backtesting Tests for Value-at-risk Models," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 9(1), pages 132-161, Winter.
    25. Francq, Christian & Zako an, Jean-Michel, 2006. "Mixing Properties Of A General Class Of Garch(1,1) Models Without Moment Assumptions On The Observed Process," Econometric Theory, Cambridge University Press, vol. 22(05), pages 815-834, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. repec:gam:jeners:v:11:y:2018:i:1:p:193-:d:126758 is not listed on IDEAS
    2. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    3. repec:eco:journ1:2018-03-6 is not listed on IDEAS
    4. repec:eee:eneeco:v:66:y:2017:i:c:p:523-534 is not listed on IDEAS
    5. Ziggel, Daniel & Berens, Tobias & Weiß, Gregor N.F. & Wied, Dominik, 2014. "A new set of improved Value-at-Risk backtests," Journal of Banking & Finance, Elsevier, vol. 48(C), pages 29-41.
    6. Guillén, Montserrat & Sarabia, José María & Prieto, Faustino, 2013. "Simple risk measure calculations for sums of positive random variables," Insurance: Mathematics and Economics, Elsevier, vol. 53(1), pages 273-280.
    7. repec:eee:jbrese:v:89:y:2018:i:c:p:216-222 is not listed on IDEAS
    8. Juan Carlos Escanciano & Zaichao Du, 2015. "Backtesting Expected Shortfall: Accounting for Tail Risk," Caepr Working Papers 2015-001, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
    9. repec:gam:jrisks:v:6:y:2018:i:2:p:61-:d:150249 is not listed on IDEAS
    10. Nieto, Maria Rosa & Ruiz, Esther, 2016. "Frontiers in VaR forecasting and backtesting," International Journal of Forecasting, Elsevier, vol. 32(2), pages 475-501.
    11. Gregor Wei{ss} & Marcus Scheffer, 2012. "Smooth Nonparametric Bernstein Vine Copulas," Papers 1210.2043, arXiv.org.

    More about this item

    JEL classification:

    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C32 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Time-Series Models; Dynamic Quantile Regressions; Dynamic Treatment Effect Models; Diffusion Processes; State Space Models
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages
    • G32 - Financial Economics - - Corporate Finance and Governance - - - Financing Policy; Financial Risk and Risk Management; Capital and Ownership Structure; Value of Firms; Goodwill

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inu:caeprp:2012-003. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Center for Applied Economics and Policy Research). General contact details of provider: http://edirc.repec.org/data/caeprus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.