Pitfalls in Backtesting Historical Simulation VaR Models
Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most widely used Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our ndings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical ndings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. Finally, our theoretical ndings are con rmed in a Monte Carlo simulation study and an empirical application with three U.S. stocks. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.
|Date of creation:||Feb 2012|
|Contact details of provider:|| Postal: 812-855-1021|
Web page: http://www.iub.edu/~caepr
More information through EDIRC
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value-at-Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, 06.
- Juan Carlos Escanciano & Jose Olmo, 2007.
"Backtesting Parametric Value-at-Risk with Estimation Risk,"
Caepr Working Papers
2007-005_updated, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
- Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
- Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011.
"Evaluating Value-at-Risk Models with Desk-Level Data,"
INFORMS, vol. 57(12), pages 2213-2227, December.
- Peter Christoffersen & Jeremy Berkowitz & Denis Pelletier, 2008. "Evaluating Value-at-Risk Models with Desk-Level Data," CREATES Research Papers 2009-35, Department of Economics and Business Economics, Aarhus University.
- Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2005. "Evaluating Value-at-Risk models with desk-level data," Working Paper Series 010, North Carolina State University, Department of Economics, revised Dec 2006.
- Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(01), pages 17-39, February.
- Joan Jasiak & C. Gourieroux, 2006.
"Dynamic Quantile Models,"
2006_4, York University, Department of Economics.
- Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
- M.J.B. Hall, 1996.
"The amendment to the capital accord to incorporate market risk,"
BNL Quarterly Review,
Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
- M.J.B. Hall, 1996. "The amendment to the capital accord to incorporate market risk," Banca Nazionale del Lavoro Quarterly Review, Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
- Robert F. Engle & Simone Manganelli, 2004.
"CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles,"
Journal of Business & Economic Statistics,
American Statistical Association, vol. 22, pages 367-381, October.
- Robert Engle & Simone Manganelli, 2000. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Econometric Society World Congress 2000 Contributed Papers 0841, Econometric Society.
- Engle, Robert F & Manganelli, Simone, 1999. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," University of California at San Diego, Economics Working Paper Series qt06m3d6nv, Department of Economics, UC San Diego.
- Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, Elsevier.
- Peter Christoffersen, 2004.
"Backtesting Value-at-Risk: A Duration-Based Approach,"
Journal of Financial Econometrics,
Society for Financial Econometrics, vol. 2(1), pages 84-108.
- Peter Christoffersen & Denis Pelletier, 2003. "Backtesting Value-at-Risk: A Duration-Based Approach," CIRANO Working Papers 2003s-05, CIRANO.
- Li, W K & Ling, Shiqing & McAleer, Michael, 2002. " Recent Theoretical Results for Time Series Models with GARCH Errors," Journal of Economic Surveys, Wiley Blackwell, vol. 16(3), pages 245-269, July.
- Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, number 8355.
- Koenker, Roger & Xiao, Zhijie, 2006. "Quantile Autoregression," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 980-990, September.
- Francq, Christian & Zako an, Jean-Michel, 2006. "Mixing Properties Of A General Class Of Garch(1,1) Models Without Moment Assumptions On The Observed Process," Econometric Theory, Cambridge University Press, vol. 22(05), pages 815-834, October.
- Sean D. Campbell, 2005. "A review of backtesting and backtesting procedures," Finance and Economics Discussion Series 2005-21, Board of Governors of the Federal Reserve System (U.S.).
- Yoshihara, Ken-ichi, 1995. "The Bahadur representation of sample quantiles for sequences of strongly mixing random variables," Statistics & Probability Letters, Elsevier, vol. 24(4), pages 299-304, September.
- Hansen, B.E., 1992.
"Autoregressive Conditional Density Estimation,"
RCER Working Papers
322, University of Rochester - Center for Economic Research (RCER).
- J. Carlos Escanciano & Jose Olmo, 2011. "Robust Backtesting Tests for Value-at-risk Models," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 9(1), pages 132-161, Winter.
- Sun, Shuxia, 2006. "The Bahadur representation for sample quantiles under weak dependence," Statistics & Probability Letters, Elsevier, vol. 76(12), pages 1238-1244, July.
- Pritsker, Matthew, 2006. "The hidden dangers of historical simulation," Journal of Banking & Finance, Elsevier, vol. 30(2), pages 561-582, February.
- Delgado, Miguel A. & Carlos Escanciano, J., 2007. "Nonparametric tests for conditional symmetry in dynamic models," Journal of Econometrics, Elsevier, vol. 141(2), pages 652-682, December.
- Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-474, October.
- Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-862, November.
- Pérignon, Christophe & Smith, Daniel R., 2010. "The level and quality of Value-at-Risk disclosure by commercial banks," Journal of Banking & Finance, Elsevier, vol. 34(2), pages 362-377, February.
When requesting a correction, please mention this item's handle: RePEc:inu:caeprp:2012-003. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Center for Applied Economics and Policy Research)
If references are entirely missing, you can add them using this form.