IDEAS home Printed from https://ideas.repec.org/a/eee/finsta/v23y2016icp79-91.html
   My bibliography  Save this article

Model risk of risk models

Author

Listed:
  • Danielsson, Jon
  • James, Kevin R.
  • Valenzuela, Marcela
  • Zer, Ilknur

Abstract

This paper evaluates the model risk of models used for forecasting systemic and market risk. Model risk, which is the potential for different models to provide inconsistent outcomes, is shown to be increasing with market uncertainty. During calm periods, the underlying risk forecast models produce similar risk readings; hence, model risk is typically negligible. However, the disagreement between the various candidate models increases significantly during market distress, further frustrating the reliability of risk readings. Finally, particular conclusions on the underlying reasons for the high model risk and the implications for practitioners and policy makers are discussed.

Suggested Citation

  • Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
  • Handle: RePEc:eee:finsta:v:23:y:2016:i:c:p:79-91
    DOI: 10.1016/j.jfs.2016.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1572308916000231
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jfs.2016.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Christian T. Brownlees & Giampiero M. Gallo, 2010. "Comparison of Volatility Measures: a Risk Management Perspective," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 8(1), pages 29-56, Winter.
    2. Markus K. Brunnermeier & Yuliy Sannikov, 2014. "A Macroeconomic Model with a Financial Sector," American Economic Review, American Economic Association, vol. 104(2), pages 379-421, February.
    3. Drehmann, Mathias & Tarashev, Nikola, 2013. "Measuring the systemic importance of interconnected banks," Journal of Financial Intermediation, Elsevier, vol. 22(4), pages 586-607.
    4. Darryll Hendricks, 1996. "Evaluation of value-at-risk models using historical data," Economic Policy Review, Federal Reserve Bank of New York, vol. 2(Apr), pages 39-69.
    5. Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value-at-Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, June.
    6. Hansen, Bruce E, 1994. "Autoregressive Conditional Density Estimation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 35(3), pages 705-730, August.
    7. Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
    8. James M. O'Brien & Pawel J. Szerszen, 2014. "An Evaluation of Bank VaR Measures for Market Risk During and Before the Financial Crisis," Finance and Economics Discussion Series 2014-21, Board of Governors of the Federal Reserve System (U.S.).
    9. Viral V. Acharya & Lasse H. Pedersen & Thomas Philippon & Matthew Richardson, 2017. "Measuring Systemic Risk," Review of Financial Studies, Society for Financial Studies, vol. 30(1), pages 2-47.
    10. repec:fip:fedhpr:y:2010:i:may:p:65-71 is not listed on IDEAS
    11. Ser-Huang Poon & Clive W.J. Granger, 2003. "Forecasting Volatility in Financial Markets: A Review," Journal of Economic Literature, American Economic Association, vol. 41(2), pages 478-539, June.
    12. Keith Kuester & Stefan Mittnik & Marc S. Paolella, 2006. "Value-at-Risk Prediction: A Comparison of Alternative Strategies," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 4(1), pages 53-89.
    13. Detken, Carsten & Alessi, Lucia, 2009. "'Real time'early warning indicators for costly asset price boom/bust cycles: a role for global liquidity," Working Paper Series 1039, European Central Bank.
    14. Huang, Xin & Zhou, Hao & Zhu, Haibin, 2009. "A framework for assessing the systemic risk of major financial institutions," Journal of Banking & Finance, Elsevier, vol. 33(11), pages 2036-2049, November.
    15. Suh, Sangwon, 2012. "Measuring systemic risk: A factor-augmented correlated default approach," Journal of Financial Intermediation, Elsevier, vol. 21(2), pages 341-358.
    16. Nikola Tarashev & Claudio Borio & Kostas Tsatsaronis, 2010. "Attributing systemic risk to individual institutions," BIS Working Papers 308, Bank for International Settlements.
    17. Rama Cont, 2006. "Model Uncertainty And Its Impact On The Pricing Of Derivative Instruments," Mathematical Finance, Wiley Blackwell, vol. 16(3), pages 519-547, July.
    18. Claudio Borio & Mathias Drehmann, 2009. "Assessing the risk of banking crises - revisited," BIS Quarterly Review, Bank for International Settlements, March.
    19. Bekiros, Stelios D. & Georgoutsos, Dimitris A., 2005. "Estimation of Value-at-Risk by extreme value and conventional methods: a comparative evaluation of their predictive performance," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 15(3), pages 209-228, July.
    20. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    21. Bauwens, Luc & Laurent, Sebastien, 2005. "A New Class of Multivariate Skew Densities, With Application to Generalized Autoregressive Conditional Heteroscedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 346-354, July.
    22. Jon Danielsson & Kevin R. James & Marcela Valenzuela & Ilknur Zer, 2016. "Can We Prove a Bank Guilty of Creating Systemic Risk? A Minority Report," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 48(4), pages 795-812, June.
    23. Billio, Monica & Getmansky, Mila & Lo, Andrew W. & Pelizzon, Loriana, 2012. "Econometric measures of connectedness and systemic risk in the finance and insurance sectors," Journal of Financial Economics, Elsevier, vol. 104(3), pages 535-559.
    24. T. Clifton Green & Stephen Figlewski, 1999. "Market Risk and Model Risk for a Financial Institution Writing Options," Journal of Finance, American Finance Association, vol. 54(4), pages 1465-1499, August.
    25. Charles Goodhart & Miguel Segoviano, 2009. "Banking Stability Measures," FMG Discussion Papers dp627, Financial Markets Group.
    26. Marimoutou, Velayoudoum & Raggad, Bechir & Trabelsi, Abdelwahed, 2009. "Extreme Value Theory and Value at Risk: Application to oil market," Energy Economics, Elsevier, vol. 31(4), pages 519-530, July.
    27. Segoviano, Miguel A. & Goodhart, Charles, 2009. "Banking stability measures," LSE Research Online Documents on Economics 24416, London School of Economics and Political Science, LSE Library.
    28. Danielsson, Jon & Morimoto, Yuji, 2000. "Forecasting Extreme Financial Risk: A Critical Analysis of Practical Methods for the Japanese Market," Monetary and Economic Studies, Institute for Monetary and Economic Studies, Bank of Japan, vol. 18(2), pages 25-48, December.
    29. Mr. Andreas A. Jobst & Mr. Dale F Gray, 2013. "Systemic Contingent Claims Analysis: Estimating Market-Implied Systemic Risk," IMF Working Papers 2013/054, International Monetary Fund.
    30. Huang, Xin & Zhou, Hao & Zhu, Haibin, 2012. "Assessing the systemic risk of a heterogeneous portfolio of banks during the recent financial crisis," Journal of Financial Stability, Elsevier, vol. 8(3), pages 193-205.
    31. Dimitrios Bisias & Mark Flood & Andrew W. Lo & Stavros Valavanis, 2012. "A Survey of Systemic Risk Analytics," Annual Review of Financial Economics, Annual Reviews, vol. 4(1), pages 255-296, October.
    32. Darryll Hendricks, 1996. "Evaluation of value-at-risk models using historical data," Proceedings 512, Federal Reserve Bank of Chicago.
    33. Bluhm, Marcel & Krahnen, Jan Pieter, 2014. "Systemic risk in an interconnected banking system with endogenous asset markets," Journal of Financial Stability, Elsevier, vol. 13(C), pages 75-94.
    34. Danielsson, Jon, 2002. "The emperor has no clothes: Limits to risk modelling," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1273-1296, July.
    35. Turan Bali & Panayiotis Theodossiou, 2007. "A conditional-SGT-VaR approach with alternative GARCH models," Annals of Operations Research, Springer, vol. 151(1), pages 241-267, April.
    36. Hull, John & Suo, Wulin, 2002. "A Methodology for Assessing Model Risk and its Application to the Implied Volatility Function Model," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 37(2), pages 297-318, June.
    37. Girardi, Giulio & Tolga Ergün, A., 2013. "Systemic risk measurement: Multivariate GARCH estimation of CoVaR," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 3169-3180.
    38. Georg Mainik & Eric Schaanning, 2012. "On dependence consistency of CoVaR and some other systemic risk measures," Papers 1207.3464, arXiv.org, revised Aug 2012.
    39. Rama Cont, 2006. "Model uncertainty and its impact on the pricing of derivative instruments," Post-Print halshs-00002695, HAL.
    40. Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-350, July.
    41. Viral Acharya & Robert Engle & Matthew Richardson, 2012. "Capital Shortfall: A New Approach to Ranking and Regulating Systemic Risks," American Economic Review, American Economic Association, vol. 102(3), pages 59-64, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Silva, Walmir & Kimura, Herbert & Sobreiro, Vinicius Amorim, 2017. "An analysis of the literature on systemic financial risk: A survey," Journal of Financial Stability, Elsevier, vol. 28(C), pages 91-114.
    2. Drakos, Anastassios A. & Kouretas, Georgios P., 2015. "Bank ownership, financial segments and the measurement of systemic risk: An application of CoVaR," International Review of Economics & Finance, Elsevier, vol. 40(C), pages 127-140.
    3. Reboredo, Juan C. & Ugolini, Andrea, 2015. "A vine-copula conditional value-at-risk approach to systemic sovereign debt risk for the financial sector," The North American Journal of Economics and Finance, Elsevier, vol. 32(C), pages 98-123.
    4. Varotto, Simone & Zhao, Lei, 2018. "Systemic risk and bank size," Journal of International Money and Finance, Elsevier, vol. 82(C), pages 45-70.
    5. Antonio Di Cesare & Anna Rogantini Picco, 2018. "A Survey of Systemic Risk Indicators," Questioni di Economia e Finanza (Occasional Papers) 458, Bank of Italy, Economic Research and International Relations Area.
    6. Reboredo, Juan C. & Ugolini, Andrea, 2015. "Systemic risk in European sovereign debt markets: A CoVaR-copula approach," Journal of International Money and Finance, Elsevier, vol. 51(C), pages 214-244.
    7. Kamila Sommer, 2014. "Fertility Choice in a Life Cycle Model with Idiosyncratic Uninsurable Earnings Risk," Finance and Economics Discussion Series 2014-32, Board of Governors of the Federal Reserve System (U.S.).
    8. Girardi, Giulio & Tolga Ergün, A., 2013. "Systemic risk measurement: Multivariate GARCH estimation of CoVaR," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 3169-3180.
    9. Carol Alexander & José María Sarabia, 2012. "Quantile Uncertainty and Value‐at‐Risk Model Risk," Risk Analysis, John Wiley & Sons, vol. 32(8), pages 1293-1308, August.
    10. Ebrahimi Kahou, Mahdi & Lehar, Alfred, 2017. "Macroprudential policy: A review," Journal of Financial Stability, Elsevier, vol. 29(C), pages 92-105.
    11. Idier, Julien & Lamé, Gildas & Mésonnier, Jean-Stéphane, 2014. "How useful is the Marginal Expected Shortfall for the measurement of systemic exposure? A practical assessment," Journal of Banking & Finance, Elsevier, vol. 47(C), pages 134-146.
    12. Zhang, Weiping & Zhuang, Xintian & Wang, Jian & Lu, Yang, 2020. "Connectedness and systemic risk spillovers analysis of Chinese sectors based on tail risk network," The North American Journal of Economics and Finance, Elsevier, vol. 54(C).
    13. Carol Alexander & Jose Maria Sarabia, 2010. "Endogenizing Model Risk to Quantile Estimates," ICMA Centre Discussion Papers in Finance icma-dp2010-07, Henley Business School, University of Reading.
    14. Bernal, Oscar & Gnabo, Jean-Yves & Guilmin, Grégory, 2014. "Assessing the contribution of banks, insurance and other financial services to systemic risk," Journal of Banking & Finance, Elsevier, vol. 47(C), pages 270-287.
    15. Van Cauwenberge, Annelies & Vancauteren, Mark & Braekers, Roel & Vandemaele, Sigrid, 2019. "International trade, foreign direct investments, and firms’ systemic risk : Evidence from the Netherlands," Economic Modelling, Elsevier, vol. 81(C), pages 361-386.
    16. Castro, Carlos & Ferrari, Stijn, 2014. "Measuring and testing for the systemically important financial institutions," Journal of Empirical Finance, Elsevier, vol. 25(C), pages 1-14.
    17. Kubitza, Christian & Gründl, Helmut, 2016. "Systemic risk: Time-lags and persistence," ICIR Working Paper Series 20/16, Goethe University Frankfurt, International Center for Insurance Regulation (ICIR).
    18. Dungey, Mardi & Luciani, Matteo & Veredas, David, 2018. "Systemic risk in the US: Interconnectedness as a circuit breaker," Economic Modelling, Elsevier, vol. 71(C), pages 305-315.
    19. Farkas, Walter & Fringuellotti, Fulvia & Tunaru, Radu, 2020. "A cost-benefit analysis of capital requirements adjusted for model risk," Journal of Corporate Finance, Elsevier, vol. 65(C).
    20. Arnold, Bruce & Borio, Claudio & Ellis, Luci & Moshirian, Fariborz, 2012. "Systemic risk, macroprudential policy frameworks, monitoring financial systems and the evolution of capital adequacy," Journal of Banking & Finance, Elsevier, vol. 36(12), pages 3125-3132.

    More about this item

    Keywords

    Model risk; Systemic risk; Value-at-Risk; Expected shortfall; Basel III;
    All these keywords.

    JEL classification:

    • G01 - Financial Economics - - General - - - Financial Crises
    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)
    • G18 - Financial Economics - - General Financial Markets - - - Government Policy and Regulation
    • G20 - Financial Economics - - Financial Institutions and Services - - - General
    • G28 - Financial Economics - - Financial Institutions and Services - - - Government Policy and Regulation
    • G38 - Financial Economics - - Corporate Finance and Governance - - - Government Policy and Regulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finsta:v:23:y:2016:i:c:p:79-91. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.elsevier.com/locate/jfstabil .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jfstabil .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.