IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/62194.html
   My bibliography  Save this paper

Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network

Author

Listed:
  • Alessi, Lucia
  • Antunes, Antonio
  • Babecky, Jan
  • Baltussen, Simon
  • Behn, Markus
  • Bonfim, Diana
  • Bush, Oliver
  • Detken, Carsten
  • Frost, Jon
  • Guimaraes, Rodrigo
  • Havranek, Tomas
  • Joy, Mark
  • Kauko, Karlo
  • Mateju, Jakub
  • Monteiro, Nuno
  • Neudorfer, Benjamin
  • Peltonen, Tuomas
  • Rodrigues, Paulo
  • Rusnak, Marek
  • Schudel, Willem
  • Sigmund, Michael
  • Stremmel, Hanno
  • Smidkova, Katerina
  • van Tilburg, Ruben
  • Vasicek, Borek
  • Zigraiova, Diana

Abstract

Over the recent decades researchers in academia and central banks have developed early warning systems (EWS) designed to warn policy makers of potential future economic and financial crises. These EWS are based on diverse approaches and empirical models. In this paper we compare the performance of nine distinct models for predicting banking crises resulting from the work of the Macroprudential Research Network (MaRs) initiated by the European System of Central Banks. In order to ensure comparability, all models use the same database of crises created by MaRs and comparable sets of potential early warning indicators. We evaluate the models’ relative usefulness by comparing the ratios of false alarms and missed crises and discuss implications for pratical use and future research. We find that multivariate models, in their many appearances, have great potential added value over simple signalling models. One of the main policy recommendations coming from this exercise is that policy makers can benefit from taking a broad methodological approach when they develop models to set macro-prudential instruments.

Suggested Citation

  • Alessi, Lucia & Antunes, Antonio & Babecky, Jan & Baltussen, Simon & Behn, Markus & Bonfim, Diana & Bush, Oliver & Detken, Carsten & Frost, Jon & Guimaraes, Rodrigo & Havranek, Tomas & Joy, Mark & Kau, 2015. "Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network," MPRA Paper 62194, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:62194
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/62194/1/MPRA_paper_62194.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Eichengreen, Barry & Rose, Andrew K & Wyplosz, Charles, 1996. "Contagious Currency Crises," CEPR Discussion Papers 1453, C.E.P.R. Discussion Papers.
    2. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    3. David Aikman & Piergiorgio Alessandri & Bruno Eklund & Prasanna Gai & Sujit Kapadia & Elizabeth Martin & Nada Mora & Gabriel Sterne & Matthew Willison, 2011. "Funding Liquidity Risk in a Quantitative Model of Systemic Stability," Central Banking, Analysis, and Economic Policies Book Series, in: Rodrigo Alfaro (ed.),Financial Stability, Monetary Policy, and Central Banking, edition 1, volume 15, chapter 12, pages 371-410, Central Bank of Chile.
    4. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    5. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    6. Luc Laeven & Fabian Valencia, 2020. "Systemic Banking Crises Database II," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 68(2), pages 307-361, June.
    7. Andrew K. Rose & Mark M. Spiegel, "undated". "Cross-Country Causes and Consequences of the 2008 Crisis: Early Warning," Working Papers 6, Department of the Treasury, Ministry of the Economy and of Finance.
    8. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency Crashes in Emerging Markets: Empirical Indicators," Center for International and Development Economics Research (CIDER) Working Papers 233424, University of California-Berkeley, Department of Economics.
    9. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    10. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    11. Krugman, Paul, 1979. "A Model of Balance-of-Payments Crises," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 11(3), pages 311-325, August.
    12. Fabian Valencia & Luc Laeven, 2008. "Systemic Banking Crises; A New Database," IMF Working Papers 2008/224, International Monetary Fund.
    13. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    14. repec:erf:erfstu:78 is not listed on IDEAS
    15. Asli Demirgüç-Kunt & Enrica Detragiache, 2005. "Cross-Country Empirical Studies of Systemic Bank Distress: A Survey," National Institute Economic Review, National Institute of Economic and Social Research, vol. 192(1), pages 68-83, April.
    16. Helmut Elsinger & Alfred Lehar & Martin Summer, 2006. "Using Market Information for Banking System Risk Assessment," International Journal of Central Banking, International Journal of Central Banking, vol. 2(1), March.
    17. Davis, E. Philip & Karim, Dilruba, 2008. "Comparing early warning systems for banking crises," Journal of Financial Stability, Elsevier, vol. 4(2), pages 89-120, June.
    18. Forbes, Kristin J. & Warnock, Francis E., 2012. "Capital flow waves: Surges, stops, flight, and retrenchment," Journal of International Economics, Elsevier, vol. 88(2), pages 235-251.
    19. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency crashes in emerging markets: An empirical treatment," Journal of International Economics, Elsevier, vol. 41(3-4), pages 351-366, November.
    20. Lilien, David M, 1982. "Sectoral Shifts and Cyclical Unemployment," Journal of Political Economy, University of Chicago Press, vol. 90(4), pages 777-793, August.
    21. Bussiere, Matthieu & Fratzscher, Marcel, 2006. "Towards a new early warning system of financial crises," Journal of International Money and Finance, Elsevier, vol. 25(6), pages 953-973, October.
    22. António R. Antunes & Diana Bonfim & Nuno Monteiro & Paulo M.M. Rodrigues, 2014. "Early Warning Indicators of Banking Crises: Exploring new Data and Tools," Economic Bulletin and Financial Stability Report Articles and Banco de Portugal Economic Studies, Banco de Portugal, Economics and Research Department.
    23. Alejandro Gaytán & Christian A. Johnson, 2002. "A Review of the Literature on Early Warning Systems for Banking Crises," Working Papers Central Bank of Chile 183, Central Bank of Chile.
    24. Abdul d Abiad, 2003. "Early Warning Systems; A Survey and a Regime-Switching Approach," IMF Working Papers 2003/032, International Monetary Fund.
    25. Alessi, Lucia, 2014. "Identifying Excessive Credit Growth and Leverage," Financial Stability Review, European Central Bank, vol. 1.
    26. Jeffrey A. Frankel & George Saravelos, 2010. "Are Leading Indicators of Financial Crises Useful for Assessing Country Vulnerability? Evidence from the 2008-09 Global Crisis," NBER Working Papers 16047, National Bureau of Economic Research, Inc.
    27. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mathonnat, Clément & Minea, Alexandru, 2018. "Financial development and the occurrence of banking crises," Journal of Banking & Finance, Elsevier, vol. 96(C), pages 344-354.
    2. Niall Conroy & Eddie Casey, 2019. "The Current Account, a Real-Time Signal of Economic Imbalances or 20/20 Hindsight?," The Economic and Social Review, Economic and Social Studies, vol. 50(1), pages 77-102.
    3. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).
    4. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    5. Berlinger, Edina, 2017. "Implicit rating: A potential new method to alert crisis on the interbank lending market," Finance Research Letters, Elsevier, vol. 21(C), pages 277-283.
    6. Tölö, Eero, 2019. "Predicting systemic financial crises with recurrent neural networks," Research Discussion Papers 14/2019, Bank of Finland.
    7. Maria Siranova & Marek Radvanský, 2018. "Performance of the Macroeconomic Imbalance Procedure in light of historical experience in the CEE region," Journal of Economic Policy Reform, Taylor & Francis Journals, vol. 21(4), pages 335-352, October.
    8. Alessandro Bitetto & Paola Cerchiello & Charilaos Mertzanis, 2021. "A data-driven approach to measuring financial soundness throughout the world," DEM Working Papers Series 199, University of Pavia, Department of Economics and Management.
    9. Ons Jedidi & Jean-Sébastien Pentecôte, 2015. "Prédire les crises bancaires : un système d’alerte robuste," Revue française d'économie, Presses de Sciences-Po, vol. 0(3), pages 189-225.
    10. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    11. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    12. Ermanno Catullo & Antonio Palestrini & Ruggero Grilli & Mauro Gallegati, 2018. "Early warning indicators and macro-prudential policies: a credit network agent based model," Journal of Economic Interaction and Coordination, Springer;Society for Economic Science with Heterogeneous Interacting Agents, vol. 13(1), pages 81-115, April.
    13. Domonkos Tomáš & Ostrihoň Filip & Šikulová Ivana & Širaňová Mária, 2017. "Analysing the Relevance of the MIP Scoreboard's Indicators," National Institute Economic Review, National Institute of Economic and Social Research, vol. 239(1), pages 32-52, February.
    14. Lukasz, Rachel & Fisher, Jack, 2016. "Assessing vulnerabilities to financial shocks in some key global economies," Bank of England working papers 636, Bank of England.
    15. Caggiano, Giovanni & Calice, Pietro & Leonida, Leone & Kapetanios, George, 2016. "Comparing logit-based early warning systems: Does the duration of systemic banking crises matter?," Journal of Empirical Finance, Elsevier, vol. 37(C), pages 104-116.
    16. Gernát, Peter & Košťálová, Zuzana & Lyócsa, Štefan, 2020. "What drives U.S. financial sector volatility? A Bayesian model averaging perspective," Research in International Business and Finance, Elsevier, vol. 51(C).
    17. Figini, Silvia & Maggi, Mario & Uberti, Pierpaolo, 2020. "The market rank indicator to detect financial distress," Econometrics and Statistics, Elsevier, vol. 14(C), pages 63-73.
    18. Kollar, Miroslav & Schmieder, Christian, 2019. "Macro-based asset allocation: An empirical analysis," EIB Working Papers 2019/11, European Investment Bank (EIB).
    19. Fabrizio Ferriani & Wanda Cornacchia & Paolo Farroni & Eliana Ferrara & Francesco Guarino & Francesco Pisanti, 2019. "An early warning system for less significant Italian banks," Questioni di Economia e Finanza (Occasional Papers) 480, Bank of Italy, Economic Research and International Relations Area.
    20. Mathias Drehmann & Anamaria Illes & Mikael Juselius & Marjorie Santos, 2015. "How much income is used for debt payments? A new database for debt service ratios," BIS Quarterly Review, Bank for International Settlements, September.
    21. Virtanen, Timo & Tölö, Eero & Virén, Matti & Taipalus, Katja, 2018. "Can bubble theory foresee banking crises?," Journal of Financial Stability, Elsevier, vol. 36(C), pages 66-81.
    22. Filippopoulou, Chryssanthi & Galariotis, Emilios & Spyrou, Spyros, 2020. "An early warning system for predicting systemic banking crises in the Eurozone: A logit regression approach," Journal of Economic Behavior & Organization, Elsevier, vol. 172(C), pages 344-363.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Christofides, Charis & Eicher, Theo S. & Papageorgiou, Chris, 2016. "Did established Early Warning Signals predict the 2008 crises?," European Economic Review, Elsevier, vol. 81(C), pages 103-114.
    2. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    3. Schudel, Willem, 2015. "Shifting horizons: assessing macro trends before, during, and following systemic banking crises," Working Paper Series 1766, European Central Bank.
    4. V. Coudert & J. Idier, 2016. "An Early Warning System for Macro-prudential Policy in France," Working papers 609, Banque de France.
    5. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    6. Sondermann, David & Zorell, Nico, 2019. "A macroeconomic vulnerability model for the euro area," Working Paper Series 2306, European Central Bank.
    7. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    8. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2013. "Leading indicators of crisis incidence: Evidence from developed countries," Journal of International Money and Finance, Elsevier, vol. 35(C), pages 1-19.
    9. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    10. Fendel Ralf & Stremmel Hanno, 2016. "Characteristics of Banking Crises: A Comparative Study with Geographical Contagion," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(3), pages 349-388, May.
    11. Stijn Claessens & M. Ayhan Kose, 2013. "Financial Crises: Explanations, Types and Implications," CAMA Working Papers 2013-06, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    12. Antunes, António & Bonfim, Diana & Monteiro, Nuno & Rodrigues, Paulo M.M., 2018. "Forecasting banking crises with dynamic panel probit models," International Journal of Forecasting, Elsevier, vol. 34(2), pages 249-275.
    13. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
    14. Lainà, Patrizio & Nyholm, Juho & Sarlin, Peter, 2015. "Leading indicators of systemic banking crises: Finland in a panel of EU countries," Review of Financial Economics, Elsevier, vol. 24(C), pages 18-35.
    15. Catão, Luis A.V. & Milesi-Ferretti, Gian Maria, 2014. "External liabilities and crises," Journal of International Economics, Elsevier, vol. 94(1), pages 18-32.
    16. Jan Babecky & Tomas Havranek & Jakub Mateju & Marek Rusnak & Katerina Smidkova & Borek Vasicek, 2011. "Early Warning Indicators of Economic Crises: Evidence from a Panel of 40 Developed Countries," Working Papers 2011/08, Czech National Bank.
    17. Victor Yotzov, 2014. "Prognostic Power of Early Warning Signals for Financial Crises – Theoretical Approaches and Empirical Results," Economic Studies journal, Bulgarian Academy of Sciences - Economic Research Institute, issue 2, pages 3-38.
    18. Hamdaoui, Mekki, 2016. "Are systemic banking crises in developed and developing countries predictable?," Journal of Multinational Financial Management, Elsevier, vol. 37, pages 114-138.
    19. Bordo, M.D. & Meissner, C.M., 2016. "Fiscal and Financial Crises," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 355-412, Elsevier.
    20. Figini, Silvia & Maggi, Mario & Uberti, Pierpaolo, 2020. "The market rank indicator to detect financial distress," Econometrics and Statistics, Elsevier, vol. 14(C), pages 63-73.

    More about this item

    Keywords

    Early warning systems; financial crisis; Bayesian model averaging;
    All these keywords.

    JEL classification:

    • G01 - Financial Economics - - General - - - Financial Crises

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:62194. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Joachim Winter). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.