Pitfalls in backtesting Historical Simulation VaR models
Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most popular Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our findings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical findings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. A Monte Carlo study and an empirical application with three US stocks confirm our theoretical findings. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- J. Carlos Escanciano & Jose Olmo, 2011. "Robust Backtesting Tests for Value-at-risk Models," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 9(1), pages 132-161, Winter.
- Hansen, B.E., 1992.
"Autoregressive Conditional Density Estimation,"
RCER Working Papers
322, University of Rochester - Center for Economic Research (RCER).
- Peter Christoffersen & Jeremy Berkowitz & Denis Pelletier, 2008.
"Evaluating Value-at-Risk Models with Desk-Level Data,"
CREATES Research Papers
2009-35, Department of Economics and Business Economics, Aarhus University.
- Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011. "Evaluating Value-at-Risk Models with Desk-Level Data," Management Science, INFORMS, vol. 57(12), pages 2213-2227, December.
- Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2005. "Evaluating Value-at-Risk models with desk-level data," Working Paper Series 010, North Carolina State University, Department of Economics, revised Dec 2006.
- Escanciano, J. Carlos & Olmo, Jose, 2010.
"Backtesting Parametric Value-at-Risk With Estimation Risk,"
Journal of Business & Economic Statistics,
American Statistical Association, vol. 28(1), pages 36-51.
- Juan Carlos Escanciano & Jose Olmo, 2007. "Backtesting Parametric Value-at-Risk with Estimation Risk," Caepr Working Papers 2007-005_updated, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
- Yoshihara, Ken-ichi, 1995. "The Bahadur representation of sample quantiles for sequences of strongly mixing random variables," Statistics & Probability Letters, Elsevier, vol. 24(4), pages 299-304, September.
- Francq, Christian & Zako an, Jean-Michel, 2006. "Mixing Properties Of A General Class Of Garch(1,1) Models Without Moment Assumptions On The Observed Process," Econometric Theory, Cambridge University Press, vol. 22(05), pages 815-834, October.
- Engle, Robert F & Manganelli, Simone, 1999.
"CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles,"
University of California at San Diego, Economics Working Paper Series
qt06m3d6nv, Department of Economics, UC San Diego.
- Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
- Robert Engle & Simone Manganelli, 2000. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Econometric Society World Congress 2000 Contributed Papers 0841, Econometric Society.
- Pritsker, Matthew, 2006. "The hidden dangers of historical simulation," Journal of Banking & Finance, Elsevier, vol. 30(2), pages 561-582, February.
- Gourieroux, C. & Jasiak, J., 2008.
"Dynamic quantile models,"
Journal of Econometrics,
Elsevier, vol. 147(1), pages 198-205, November.
- Pérignon, Christophe & Smith, Daniel R., 2010. "The level and quality of Value-at-Risk disclosure by commercial banks," Journal of Banking & Finance, Elsevier, vol. 34(2), pages 362-377, February.
- Sean D. Campbell, 2005. "A review of backtesting and backtesting procedures," Finance and Economics Discussion Series 2005-21, Board of Governors of the Federal Reserve System (U.S.).
- M.J.B. Hall, 1996.
"The amendment to the capital accord to incorporate market risk,"
BNL Quarterly Review,
Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
- M.J.B. Hall, 1996. "The amendment to the capital accord to incorporate market risk," Banca Nazionale del Lavoro Quarterly Review, Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
- Delgado, Miguel A. & Carlos Escanciano, J., 2007. "Nonparametric tests for conditional symmetry in dynamic models," Journal of Econometrics, Elsevier, vol. 141(2), pages 652-682, December.
- Peter Christoffersen, 2004.
"Backtesting Value-at-Risk: A Duration-Based Approach,"
Journal of Financial Econometrics,
Society for Financial Econometrics, vol. 2(1), pages 84-108.
- Peter Christoffersen & Denis Pelletier, 2003. "Backtesting Value-at-Risk: A Duration-Based Approach," CIRANO Working Papers 2003s-05, CIRANO.
- Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, number 8355, 01-2013.
- Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
- Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(01), pages 17-39, February.
- Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-74, October.
- Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, Elsevier.
- Li, W K & Ling, Shiqing & McAleer, Michael, 2002. " Recent Theoretical Results for Time Series Models with GARCH Errors," Journal of Economic Surveys, Wiley Blackwell, vol. 16(3), pages 245-69, July.
- Koenker, Roger & Xiao, Zhijie, 2006. "Quantile Autoregression," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 980-990, September.
- Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value-at-Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, 06.
- Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
- Sun, Shuxia, 2006. "The Bahadur representation for sample quantiles under weak dependence," Statistics & Probability Letters, Elsevier, vol. 76(12), pages 1238-1244, July.
When requesting a correction, please mention this item's handle: RePEc:eee:jbfina:v:36:y:2012:i:8:p:2233-2244. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Shamier, Wendy)
If references are entirely missing, you can add them using this form.