IDEAS home Printed from https://ideas.repec.org/p/ehl/lserod/66365.html
   My bibliography  Save this paper

Model risk of risk models

Author

Listed:
  • Danielsson, Jon
  • James, Kevin R.
  • Valenzuela, Marcela
  • Zer, Ilknur

Abstract

This paper evaluates the model risk of models used for forecasting systemic and market risk. Model risk, which is the potential for different models to provide inconsistent outcomes, is shown to be increasing with market uncertainty. During calm periods, the underlying risk forecast models produce similar risk readings; hence, model risk is typically negligible. However, the disagreement between the various candidate models increases significantly during market distress, further frustrating the reliability of risk readings. Finally, particular conclusions on the underlying reasons for the high model risk and the implications for practitioners and policy makers are discussed.

Suggested Citation

  • Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," LSE Research Online Documents on Economics 66365, London School of Economics and Political Science, LSE Library.
  • Handle: RePEc:ehl:lserod:66365
    as

    Download full text from publisher

    File URL: http://eprints.lse.ac.uk/66365/
    File Function: Open access version.
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Viral V. Acharya & Lasse H. Pedersen & Thomas Philippon & Matthew Richardson, 2017. "Measuring Systemic Risk," Review of Financial Studies, Society for Financial Studies, vol. 30(1), pages 2-47.
    2. Suh, Sangwon, 2012. "Measuring systemic risk: A factor-augmented correlated default approach," Journal of Financial Intermediation, Elsevier, vol. 21(2), pages 341-358.
    3. Billio, Monica & Getmansky, Mila & Lo, Andrew W. & Pelizzon, Loriana, 2012. "Econometric measures of connectedness and systemic risk in the finance and insurance sectors," Journal of Financial Economics, Elsevier, vol. 104(3), pages 535-559.
    4. Charles Goodhart & Miguel Segoviano, 2009. "Banking Stability Measures," FMG Discussion Papers dp627, Financial Markets Group.
    5. Marimoutou, Velayoudoum & Raggad, Bechir & Trabelsi, Abdelwahed, 2009. "Extreme Value Theory and Value at Risk: Application to oil market," Energy Economics, Elsevier, vol. 31(4), pages 519-530, July.
    6. Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
    7. Danielsson, Jon & Morimoto, Yuji, 2000. "Forecasting Extreme Financial Risk: A Critical Analysis of Practical Methods for the Japanese Market," Monetary and Economic Studies, Institute for Monetary and Economic Studies, Bank of Japan, vol. 18(2), pages 25-48, December.
    8. Darryll Hendricks, 1996. "Evaluation of value-at-risk models using historical data," Proceedings 512, Federal Reserve Bank of Chicago.
    9. Bluhm, Marcel & Krahnen, Jan Pieter, 2014. "Systemic risk in an interconnected banking system with endogenous asset markets," Journal of Financial Stability, Elsevier, vol. 13(C), pages 75-94.
    10. Danielsson, Jon, 2002. "The emperor has no clothes: Limits to risk modelling," Journal of Banking & Finance, Elsevier, vol. 26(7), pages 1273-1296, July.
    11. Hull, John & Suo, Wulin, 2002. "A Methodology for Assessing Model Risk and its Application to the Implied Volatility Function Model," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 37(02), pages 297-318, June.
    12. Huang, Xin & Zhou, Hao & Zhu, Haibin, 2012. "Assessing the systemic risk of a heterogeneous portfolio of banks during the recent financial crisis," Journal of Financial Stability, Elsevier, vol. 8(3), pages 193-205.
    13. Jon Danielsson & Kevin R. James & Marcela Valenzuela & Ilknur Zer, 2016. "Can We Prove a Bank Guilty of Creating Systemic Risk? A Minority Report," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 48(4), pages 795-812, June.
    14. Drehmann, Mathias & Tarashev, Nikola, 2013. "Measuring the systemic importance of interconnected banks," Journal of Financial Intermediation, Elsevier, vol. 22(4), pages 586-607.
    15. Huang, Xin & Zhou, Hao & Zhu, Haibin, 2009. "A framework for assessing the systemic risk of major financial institutions," Journal of Banking & Finance, Elsevier, vol. 33(11), pages 2036-2049, November.
    16. Segoviano, Miguel A. & Goodhart, Charles, 2009. "Banking stability measures," LSE Research Online Documents on Economics 24416, London School of Economics and Political Science, LSE Library.
    17. Dimitrios Bisias & Mark Flood & Andrew W. Lo & Stavros Valavanis, 2012. "A Survey of Systemic Risk Analytics," Annual Review of Financial Economics, Annual Reviews, vol. 4(1), pages 255-296, October.
    18. Markus K. Brunnermeier & Yuliy Sannikov, 2014. "A Macroeconomic Model with a Financial Sector," American Economic Review, American Economic Association, vol. 104(2), pages 379-421, February.
    19. Jeremy Berkowitz & James O'Brien, 2002. "How Accurate Are Value-at-Risk Models at Commercial Banks?," Journal of Finance, American Finance Association, vol. 57(3), pages 1093-1111, June.
    20. O'Brien, James M. & Szerszen, Pawel J., 2014. "An Evaluation of Bank VaR Measures for Market Risk During and Before the Financial Crisis," Finance and Economics Discussion Series 2014-21, Board of Governors of the Federal Reserve System (US).
    21. repec:fip:fedhpr:y:2010:i:may:p:65-71 is not listed on IDEAS
    22. Keith Kuester & Stefan Mittnik & Marc S. Paolella, 2006. "Value-at-Risk Prediction: A Comparison of Alternative Strategies," Journal of Financial Econometrics, Society for Financial Econometrics, vol. 4(1), pages 53-89.
    23. Bauwens, Luc & Laurent, Sebastien, 2005. "A New Class of Multivariate Skew Densities, With Application to Generalized Autoregressive Conditional Heteroscedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 23, pages 346-354, July.
    24. T. Clifton Green & Stephen Figlewski, 1999. "Market Risk and Model Risk for a Financial Institution Writing Options," Journal of Finance, American Finance Association, vol. 54(4), pages 1465-1499, August.
    25. Girardi, Giulio & Tolga Ergün, A., 2013. "Systemic risk measurement: Multivariate GARCH estimation of CoVaR," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 3169-3180.
    26. Georg Mainik & Eric Schaanning, 2012. "On dependence consistency of CoVaR and some other systemic risk measures," Papers 1207.3464, arXiv.org, revised Aug 2012.
    27. Engle, Robert, 2002. "Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(3), pages 339-350, July.
    28. Viral Acharya & Robert Engle & Matthew Richardson, 2012. "Capital Shortfall: A New Approach to Ranking and Regulating Systemic Risks," American Economic Review, American Economic Association, vol. 102(3), pages 59-64, May.
    29. Darryll Hendricks, 1996. "Evaluation of value-at-risk models using historical data," Economic Policy Review, Federal Reserve Bank of New York, issue Apr, pages 39-69.
    30. Hansen, Bruce E, 1994. "Autoregressive Conditional Density Estimation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 35(3), pages 705-730, August.
    31. Ser-Huang Poon & Clive W.J. Granger, 2003. "Forecasting Volatility in Financial Markets: A Review," Journal of Economic Literature, American Economic Association, vol. 41(2), pages 478-539, June.
    32. Detken, Carsten & Alessi, Lucia, 2009. "'Real time'early warning indicators for costly asset price boom/bust cycles: a role for global liquidity," Working Paper Series 1039, European Central Bank.
    33. Nikola Tarashev & Claudio Borio & Kostas Tsatsaronis, 2010. "Attributing systemic risk to individual institutions," BIS Working Papers 308, Bank for International Settlements.
    34. Rama Cont, 2006. "Model Uncertainty And Its Impact On The Pricing Of Derivative Instruments," Mathematical Finance, Wiley Blackwell, vol. 16(3), pages 519-547.
    35. Claudio Borio & Mathias Drehmann, 2009. "Assessing the risk of banking crises - revisited," BIS Quarterly Review, Bank for International Settlements, March.
    36. Bekiros, Stelios D. & Georgoutsos, Dimitris A., 2005. "Estimation of Value-at-Risk by extreme value and conventional methods: a comparative evaluation of their predictive performance," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 15(3), pages 209-228, July.
    37. Turan Bali & Panayiotis Theodossiou, 2007. "A conditional-SGT-VaR approach with alternative GARCH models," Annals of Operations Research, Springer, vol. 151(1), pages 241-267, April.
    38. Rama Cont, 2006. "Model uncertainty and its impact on the pricing of derivative instruments," Post-Print halshs-00002695, HAL.
    39. Andreas A. Jobst & Dale F Gray, 2013. "Systemic Contingent Claims Analysis; Estimating Market-Implied Systemic Risk," IMF Working Papers 13/54, International Monetary Fund.
    40. Viral V. Acharya, 2010. "Measuring systemic risk," Proceedings 1140, Federal Reserve Bank of Chicago.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Fernanda Maria Muller & Marcelo Brutti Righi, 2017. "A robust approach for minimization of risk measurement errors," Papers 1707.09829, arXiv.org, revised Aug 2018.
    2. repec:cnb:ocpubc:fsr1718/2 is not listed on IDEAS
    3. Zevallos, Mauricio & Villarreal, Fernanda & Del Carpio, Carlos & Abbara, Omar, 2014. "Influencia de los precios de los metales y el mercado internacional en el riesgo bursátil peruano," Working Papers 2014-023, Banco Central de Reserva del Perú.
    4. Kubitza, Christian & Gründl, Helmut, 2016. "Systemic risk: Time-lags and persistence," ICIR Working Paper Series 20/16, Goethe University Frankfurt, International Center for Insurance Regulation (ICIR).
    5. Maillet, Bertrand & Tokpavi, Sessi & Vaucher, Benoit, 2015. "Global minimum variance portfolio optimisation under some model risk: A robust regression-based approach," European Journal of Operational Research, Elsevier, vol. 244(1), pages 289-299.
    6. Boucher, Christophe M. & Daníelsson, Jón & Kouontchou, Patrick S. & Maillet, Bertrand B., 2014. "Risk models-at-risk," Journal of Banking & Finance, Elsevier, vol. 44(C), pages 72-92.
    7. Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
    8. repec:eee:jbfina:v:93:y:2018:i:c:p:213-229 is not listed on IDEAS
    9. repec:eee:ecosta:v:8:y:2018:i:c:p:56-77 is not listed on IDEAS
    10. repec:cup:jfinqa:v:53:y:2018:i:03:p:1371-1390_00 is not listed on IDEAS
    11. Pfeifer, Lukáš & Hodula, Martin, 2018. "A profit-to-provisioning approach to setting the countercyclical capital buffer: the Czech example," ESRB Working Paper Series 82, European Systemic Risk Board.
    12. Marcelo Brutti Righi, 2018. "A theory for combinations of risk measures," Papers 1807.01977, arXiv.org, revised Mar 2019.
    13. Yu Feng, 2019. "Non-Parametric Robust Model Risk Measurement with Path-Dependent Loss Functions," Papers 1903.00590, arXiv.org.
    14. Yu Feng & Ralph Rudd & Christopher Baker & Qaphela Mashalaba & Melusi Mavuso & Erik Schlogl, 2018. "Quantifying the Model Risk Inherent in the Calibration and Recalibration of Option Pricing Models," Research Paper Series 395, Quantitative Finance Research Centre, University of Technology, Sydney.
    15. repec:pcp:pucrev:y:2017:i:79:p:87-104 is not listed on IDEAS
    16. Yu Feng & Ralph Rudd & Christopher Baker & Qaphela Mashalaba & Melusi Mavuso & Erik Schlogl, 2018. "Quantifying the Model Risk Inherent in the Calibration and Recalibration of Option Pricing Models," Papers 1810.09112, arXiv.org.
    17. Bidder, Rhys & Giacomini, Raffaella & McKenna, Andrew, 2016. "Stress Testing with Misspecified Models," Working Paper Series 2016-26, Federal Reserve Bank of San Francisco.
    18. Geraci, Marco Valerio & Gnabo, Jean-Yves, 2018. "Measuring Interconnectedness between Financial Institutions with Bayesian Time-Varying Vector Autoregressions," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 53(03), pages 1371-1390, June.
    19. repec:eee:finsta:v:30:y:2017:i:c:p:126-138 is not listed on IDEAS
    20. Silva, Walmir & Kimura, Herbert & Sobreiro, Vinicius Amorim, 2017. "An analysis of the literature on systemic financial risk: A survey," Journal of Financial Stability, Elsevier, vol. 28(C), pages 91-114.
    21. repec:eee:intfor:v:34:y:2018:i:3:p:440-455 is not listed on IDEAS

    More about this item

    Keywords

    model risk; systemic risk; value-at-risk; expected shortfall; Basel III; ES/K002309/1;

    JEL classification:

    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)
    • G18 - Financial Economics - - General Financial Markets - - - Government Policy and Regulation
    • G20 - Financial Economics - - Financial Institutions and Services - - - General
    • G28 - Financial Economics - - Financial Institutions and Services - - - Government Policy and Regulation
    • G38 - Financial Economics - - Corporate Finance and Governance - - - Government Policy and Regulation

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ehl:lserod:66365. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (LSERO Manager). General contact details of provider: http://edirc.repec.org/data/lsepsuk.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.