Pitfalls in backtesting Historical Simulation VaR models
Historical Simulation (HS) and its variant, the Filtered Historical Simulation (FHS), are the most popular Value-at-Risk forecast methods at commercial banks. These forecast methods are traditionally evaluated by means of the unconditional backtest. This paper formally shows that the unconditional backtest is always inconsistent for backtesting HS and FHS models, with a power function that can be even smaller than the nominal level in large samples. Our findings have fundamental implications in the determination of market risk capital requirements, and also explain Monte Carlo and empirical findings in previous studies. We also propose a data-driven weighted backtest with good power properties to evaluate HS and FHS forecasts. A Monte Carlo study and an empirical application with three US stocks confirm our theoretical findings. The empirical application shows that multiplication factors computed under the current regulatory framework are downward biased, as they inherit the inconsistency of the unconditional backtest.
If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.
As the access to this document is restricted, you may want to look for a different version under "Related research" (further below) or search for a different version of it.
References listed on IDEAS
Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:
- Engle, Robert F & Manganelli, Simone, 1999.
"CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles,"
University of California at San Diego, Economics Working Paper Series
qt06m3d6nv, Department of Economics, UC San Diego.
- Robert F. Engle & Simone Manganelli, 2004. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Journal of Business & Economic Statistics, American Statistical Association, vol. 22, pages 367-381, October.
- Robert Engle & Simone Manganelli, 2000. "CAViaR: Conditional Autoregressive Value at Risk by Regression Quantiles," Econometric Society World Congress 2000 Contributed Papers 0841, Econometric Society.
- Christoffersen, Peter F, 1998. "Evaluating Interval Forecasts," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 39(4), pages 841-62, November.
- Juan Carlos Escanciano & Jose Olmo, 2007.
"Backtesting Parametric Value-at-Risk with Estimation Risk,"
Caepr Working Papers
2007-005_updated, Center for Applied Economics and Policy Research, Economics Department, Indiana University Bloomington.
- Escanciano, J. Carlos & Olmo, Jose, 2010. "Backtesting Parametric Value-at-Risk With Estimation Risk," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 36-51.
- Li, W K & Ling, Shiqing & McAleer, Michael, 2002. " Recent Theoretical Results for Time Series Models with GARCH Errors," Journal of Economic Surveys, Wiley Blackwell, vol. 16(3), pages 245-69, July.
- Francq, Christian & Zako an, Jean-Michel, 2006. "Mixing Properties Of A General Class Of Garch(1,1) Models Without Moment Assumptions On The Observed Process," Econometric Theory, Cambridge University Press, vol. 22(05), pages 815-834, October.
- Sean D. Campbell, 2005. "A review of backtesting and backtesting procedures," Finance and Economics Discussion Series 2005-21, Board of Governors of the Federal Reserve System (U.S.).
- Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2011.
"Evaluating Value-at-Risk Models with Desk-Level Data,"
INFORMS, vol. 57(12), pages 2213-2227, December.
- Peter Christoffersen & Jeremy Berkowitz & Denis Pelletier, 2008. "Evaluating Value-at-Risk Models with Desk-Level Data," CREATES Research Papers 2009-35, School of Economics and Management, University of Aarhus.
- Jeremy Berkowitz & Peter Christoffersen & Denis Pelletier, 2005. "Evaluating Value-at-Risk models with desk-level data," Working Paper Series 010, North Carolina State University, Department of Economics, revised Dec 2006.
- Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value-at-Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, 06.
- Hansen, Bruce E, 1994.
"Autoregressive Conditional Density Estimation,"
International Economic Review,
Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 35(3), pages 705-30, August.
- Tom Doan, . "RATS programs to replicate Hansen's GARCH models with time-varying t-densities," Statistical Software Components RTZ00086, Boston College Department of Economics.
- Hansen, B.E., 1992. "Autoregressive Conditional Density Estimation," RCER Working Papers 322, University of Rochester - Center for Economic Research (RCER).
- M.J.B. Hall, 1996. "The amendment to the capital accord to incorporate market risk," BNL Quarterly Review, Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
- Yoshihara, Ken-ichi, 1995. "The Bahadur representation of sample quantiles for sequences of strongly mixing random variables," Statistics & Probability Letters, Elsevier, vol. 24(4), pages 299-304, September.
- Koenker, Roger & Xiao, Zhijie, 2006. "Quantile Autoregression," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 980-990, September.
- M.J.B. Hall, 1996. "The amendment to the capital accord to incorporate market risk," Banca Nazionale del Lavoro Quarterly Review, Banca Nazionale del Lavoro, vol. 49(197), pages 271-277.
- Delgado, Miguel A. & Carlos Escanciano, J., 2007. "Nonparametric tests for conditional symmetry in dynamic models," Journal of Econometrics, Elsevier, vol. 141(2), pages 652-682, December.
- Peter Christoffersen, 2004.
"Backtesting Value-at-Risk: A Duration-Based Approach,"
Journal of Financial Econometrics,
Society for Financial Econometrics, vol. 2(1), pages 84-108.
- Peter Christoffersen & Denis Pelletier, 2003. "Backtesting Value-at-Risk: A Duration-Based Approach," CIRANO Working Papers 2003s-05, CIRANO.
- Gourieroux, C. & Jasiak, J., 2008.
"Dynamic quantile models,"
Journal of Econometrics,
Elsevier, vol. 147(1), pages 198-205, November.
- Sun, Shuxia, 2006. "The Bahadur representation for sample quantiles under weak dependence," Statistics & Probability Letters, Elsevier, vol. 76(12), pages 1238-1244, July.
- Paul H. Kupiec, 1995. "Techniques for verifying the accuracy of risk measurement models," Finance and Economics Discussion Series 95-24, Board of Governors of the Federal Reserve System (U.S.).
- J. Carlos Escanciano & Jose Olmo, 2011. "Robust Backtesting Tests for Value-at-risk Models," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 9(1), pages 132-161, Winter.
- Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355, March.
- Berkowitz, Jeremy, 2001. "Testing Density Forecasts, with Applications to Risk Management," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 465-74, October.
- Pritsker, Matthew, 2006. "The hidden dangers of historical simulation," Journal of Banking & Finance, Elsevier, vol. 30(2), pages 561-582, February.
- Andersen, Torben G. & Bollerslev, Tim & Christoffersen, Peter F. & Diebold, Francis X., 2006. "Volatility and Correlation Forecasting," Handbook of Economic Forecasting, Elsevier.
- Carrasco, Marine & Chen, Xiaohong, 2002. "Mixing And Moment Properties Of Various Garch And Stochastic Volatility Models," Econometric Theory, Cambridge University Press, vol. 18(01), pages 17-39, February.
- repec:cup:cbooks:9780521496032 is not listed on IDEAS
- Pérignon, Christophe & Smith, Daniel R., 2010. "The level and quality of Value-at-Risk disclosure by commercial banks," Journal of Banking & Finance, Elsevier, vol. 34(2), pages 362-377, February.
When requesting a correction, please mention this item's handle: RePEc:eee:jbfina:v:36:y:2012:i:8:p:2233-2244. See general information about how to correct material in RePEc.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Zhang, Lei)
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If references are entirely missing, you can add them using this form.
If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.
Please note that corrections may take a couple of weeks to filter through the various RePEc services.