IDEAS home Printed from https://ideas.repec.org/a/nbp/nbpbik/v56y2025i4p463-496.html

How reliable are systemic risk measures? Model risk estimates of MES and ΔCoVaR

Author

Listed:
  • Aleksandra Pasieczna-Dixit

    (Pomeranian Higher School in Starogard Gdański)

Abstract

The model risk of two systemic risk measures (SRMs) was quantified for a set of systemically important European banks, using the dispersion of SRM estimates as a proxy. A high model risk was observed, with dispersions of above 65% of the average value, associated with the parametrization error of the Monte Carlo algorithm alone, which has profound implications in the context of systemic risk. Ranking individual banks based on the SRM values was observed to become less dependable due to the high model risk of the SRMs, thus making it difficult for regulators to implement proper policies. Underestimation of the systemic risk of a bank increases the stress within the network, while overestimation of the systemic risk of a bank might lead to undue penalties levied upon the bank. The model risk metric we used additionally allowed us to rank the parameter contributions to the observed model risk.

Suggested Citation

  • Aleksandra Pasieczna-Dixit, 2025. "How reliable are systemic risk measures? Model risk estimates of MES and ΔCoVaR," Bank i Kredyt, Narodowy Bank Polski, vol. 56(4), pages 463-496.
    • Handle: RePEc:nbp:nbpbik:v:56:y:2025:i:4:p:463-496
    as

    Download full text from publisher

    File URL: https://bankikredyt.nbp.pl/content/2025/04/bik_04_2025_04.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Darryll Hendricks, 1996. "Evaluation of value-at-risk models using historical data," Economic Policy Review, Federal Reserve Bank of New York, vol. 2(Apr), pages 39-69.
    2. Castro, Carlos & Ferrari, Stijn, 2014. "Measuring and testing for the systemically important financial institutions," Journal of Empirical Finance, Elsevier, vol. 25(C), pages 1-14.
    3. Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
    4. Klein, Roger W. & Bawa, Vijay S., 1976. "The effect of estimation risk on optimal portfolio choice," Journal of Financial Economics, Elsevier, vol. 3(3), pages 215-231, June.
    5. Aleksandra Helena Pasieczna, 2021. "Model Risk of VaR and ES Using Monte Carlo: Study on Financial Institutions from Paris and Frankfurt Stock Exchanges," Springer Proceedings in Business and Economics, in: Krzysztof Jajuga & Hermann Locarek-Junge & Lucjan T. Orlowski & Karsten Staehr (ed.), Contemporary Trends and Challenges in Finance, pages 75-85, Springer.
    6. Jonathan Lewellen & Jay Shanken, 2000. "Estimation Risk, Market Efficiency, and the Predictability of Returns," NBER Working Papers 7699, National Bureau of Economic Research, Inc.
    7. Philippe Jorion, 1996. "Risk2: Measuring the Risk in Value at Risk," Financial Analysts Journal, Taylor & Francis Journals, vol. 52(6), pages 47-56, November.
    8. Jon Danielsson & Kevin R. James & Marcela Valenzuela & Ilknur Zer, 2016. "Can We Prove a Bank Guilty of Creating Systemic Risk? A Minority Report," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 48(4), pages 795-812, June.
    9. Yuhong Xu, 2014. "Robust valuation and risk measurement under model uncertainty," Papers 1407.8024, arXiv.org.
    10. Darryll Hendricks, 1996. "Evaluation of value-at-risk models using historical data," Proceedings 512, Federal Reserve Bank of Chicago.
    11. Roncoroni, Alan & Battiston, Stefano & D’Errico, Marco & Hałaj, Grzegorz & Kok, Christoffer, 2021. "Interconnected banks and systemically important exposures," Journal of Economic Dynamics and Control, Elsevier, vol. 133(C).
    12. Sylvain Benoit & Jean-Edouard Colliard & Christophe Hurlin & Christophe Pérignon, 2017. "Where the Risks Lie: A Survey on Systemic Risk," Review of Finance, European Finance Association, vol. 21(1), pages 109-152.
    13. Michele Leonardo Bianchi & Alberto Maria Sorrentino, 2020. "Measuring CoVaR: An Empirical Comparison," Computational Economics, Springer;Society for Computational Economics, vol. 55(2), pages 511-528, February.
    14. Sylvain Benoit & Jean-Edouard Colliard & Christophe Hurlin & Christophe Pérignon, 2017. "Where the Risks Lie: A Survey on Systemic Risk," Review of Finance, European Finance Association, vol. 21(1), pages 109-152.
    15. Paul Glasserman & Xingbo Xu, 2014. "Robust risk measurement and model risk," Quantitative Finance, Taylor & Francis Journals, vol. 14(1), pages 29-58, January.
    16. Filippo Curti & Ibrahim Ergen & Minh Le & Marco Migueis & Rob T. Stewart, 2016. "Benchmarking Operational Risk Models," Finance and Economics Discussion Series 2016-070, Board of Governors of the Federal Reserve System (U.S.).
    17. Löffler, Gunter & Raupach, Peter, 2018. "Pitfalls in the Use of Systemic Risk Measures," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 53(1), pages 269-298, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Moratis, Georgios & Sakellaris, Plutarchos, 2021. "Measuring the systemic importance of banks," Journal of Financial Stability, Elsevier, vol. 54(C).
    2. Mohammed Berkhouch & Fernanda Maria Müller & Ghizlane Lakhnati & Marcelo Brutti Righi, 2022. "Deviation-Based Model Risk Measures," Computational Economics, Springer;Society for Computational Economics, vol. 59(2), pages 527-547, February.
    3. Kubitza, Christian & Gründl, Helmut, 2016. "Systemic risk: Time-lags and persistence," ICIR Working Paper Series 20/16, Goethe University Frankfurt, International Center for Insurance Regulation (ICIR).
    4. Kolari, James W. & López-Iturriaga, Félix J. & Sanz, Ivan Pastor, 2020. "Measuring systemic risk in the U.S. Banking system," Economic Modelling, Elsevier, vol. 91(C), pages 646-658.
    5. Laura Garcia-Jorcano & Lidia Sanchis-Marco, 2023. "Measuring Systemic Risk Using Multivariate Quantile-Located ES Models," Journal of Financial Econometrics, Oxford University Press, vol. 21(1), pages 1-72.
    6. Denisa Banulescu-Radu & Christophe Hurlin & Jérémy Leymarie & Olivier Scaillet, 2021. "Backtesting Marginal Expected Shortfall and Related Systemic Risk Measures," Management Science, INFORMS, vol. 67(9), pages 5730-5754, September.
    7. Mehmet Sahiner & David G. McMillan & Dimos Kambouroudis, 2023. "Do artificial neural networks provide improved volatility forecasts: Evidence from Asian markets," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 47(3), pages 723-762, September.
    8. Marina Brogi & Valentina Lagasio & Luca Riccetti, 2021. "Systemic risk measurement: bucketing global systemically important banks," Annals of Finance, Springer, vol. 17(3), pages 319-351, September.
    9. Garcia-Jorcano, Laura & Sanchis-Marco, Lidia, 2021. "Systemic-systematic risk in financial system: A dynamic ranking based on expectiles," International Review of Economics & Finance, Elsevier, vol. 75(C), pages 330-365.
    10. Lazar, Emese & Zhang, Ning, 2025. "Model Risk of Volatility Models," Econometrics and Statistics, Elsevier, vol. 35(C), pages 1-22.
    11. Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
    12. Michele Leonardo Bianchi & Alberto Maria Sorrentino, 2020. "Measuring CoVaR: An Empirical Comparison," Computational Economics, Springer;Society for Computational Economics, vol. 55(2), pages 511-528, February.
    13. Bevilacqua, Mattia & Tunaru, Radu & Vioto, Davide, 2020. "Options-based systemic risk, financial distress, and macroeconomic downturns," LSE Research Online Documents on Economics 118850, London School of Economics and Political Science, LSE Library.
    14. Michele Leonardo Bianchi & Alberto Maria Sorrentino, 2022. "Exploring the Systemic Risk of Domestic Banks with ΔCoVaR and Elastic-Net," Journal of Financial Services Research, Springer;Western Finance Association, vol. 62(1), pages 127-141, October.
    15. Mariano González-Sánchez & Eva M. Ibáñez Jiménez & Ana I. Segovia San Juan, 2022. "Market and model risks: a feasible joint estimate methodology," Risk Management, Palgrave Macmillan, vol. 24(3), pages 187-213, September.
    16. Chatziantoniou, Ioannis & Colak, Gonul & Filippidis, Michail & Filis, George & Tzouvanas, Panagiotis, 2025. "Systemic risk and oil price volatility shocks," Journal of Financial Stability, Elsevier, vol. 79(C).
    17. Abendschein, Michael & Grundke, Peter, 2018. "On the ranking consistency of global systemic risk measures: empirical evidence," VfS Annual Conference 2018 (Freiburg, Breisgau): Digital Economy 181623, Verein für Socialpolitik / German Economic Association.
    18. Stepankova, Barbora & Teply, Petr, 2023. "Consistency of banks' internal probability of default estimates: Empirical evidence from the COVID-19 crisis," Journal of Banking & Finance, Elsevier, vol. 154(C).
    19. Yu Feng & Ralph Rudd & Christopher Baker & Qaphela Mashalaba & Melusi Mavuso & Erik Schlögl, 2021. "Quantifying the Model Risk Inherent in the Calibration and Recalibration of Option Pricing Models," Risks, MDPI, vol. 9(1), pages 1-20, January.
    20. Matteo Foglia & Eliana Angelini, 2021. "The triple (T3) dimension of systemic risk: Identifying systemically important banks," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(1), pages 7-26, January.

    More about this item

    Keywords

    ;
    ;
    ;
    ;
    ;

    JEL classification:

    • C63 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling - - - Computational Techniques
    • E44 - Macroeconomics and Monetary Economics - - Money and Interest Rates - - - Financial Markets and the Macroeconomy
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages
    • G28 - Financial Economics - - Financial Institutions and Services - - - Government Policy and Regulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbp:nbpbik:v:56:y:2025:i:4:p:463-496. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wojciech Burjanek (email available below). General contact details of provider: https://edirc.repec.org/data/nbpgvpl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.