IDEAS home Printed from
   My bibliography  Save this paper

Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network


  • Alessi, Lucia
  • Antunes, Antonio
  • Babecky, Jan
  • Baltussen, Simon
  • Behn, Markus
  • Bonfim, Diana
  • Bush, Oliver
  • Detken, Carsten
  • Frost, Jon
  • Guimaraes, Rodrigo
  • Havranek, Tomas
  • Joy, Mark
  • Kauko, Karlo
  • Mateju, Jakub
  • Monteiro, Nuno
  • Neudorfer, Benjamin
  • Peltonen, Tuomas
  • Rodrigues, Paulo
  • Rusnak, Marek
  • Schudel, Willem
  • Sigmund, Michael
  • Stremmel, Hanno
  • Smidkova, Katerina
  • van Tilburg, Ruben
  • Vasicek, Borek
  • Zigraiova, Diana


Over the recent decades researchers in academia and central banks have developed early warning systems (EWS) designed to warn policy makers of potential future economic and financial crises. These EWS are based on diverse approaches and empirical models. In this paper we compare the performance of nine distinct models for predicting banking crises resulting from the work of the Macroprudential Research Network (MaRs) initiated by the European System of Central Banks. In order to ensure comparability, all models use the same database of crises created by MaRs and comparable sets of potential early warning indicators. We evaluate the models’ relative usefulness by comparing the ratios of false alarms and missed crises and discuss implications for pratical use and future research. We find that multivariate models, in their many appearances, have great potential added value over simple signalling models. One of the main policy recommendations coming from this exercise is that policy makers can benefit from taking a broad methodological approach when they develop models to set macro-prudential instruments.

Suggested Citation

  • Alessi, Lucia & Antunes, Antonio & Babecky, Jan & Baltussen, Simon & Behn, Markus & Bonfim, Diana & Bush, Oliver & Detken, Carsten & Frost, Jon & Guimaraes, Rodrigo & Havranek, Tomas & Joy, Mark & Kau, 2015. "Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network," MPRA Paper 62194, University Library of Munich, Germany.
  • Handle: RePEc:pra:mprapa:62194

    Download full text from publisher

    File URL:
    File Function: original version
    Download Restriction: no

    References listed on IDEAS

    1. Eichengreen, Barry & Rose, Andrew K & Wyplosz, Charles, 1996. "Contagious Currency Crises," CEPR Discussion Papers 1453, C.E.P.R. Discussion Papers.
    2. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    3. David Aikman & Piergiorgio Alessandri & Bruno Eklund & Prasanna Gai & Sujit Kapadia & Elizabeth Martin & Nada Mora & Gabriel Sterne & Matthew Willison, 2011. "Funding Liquidity Risk in a Quantitative Model of Systemic Stability," Central Banking, Analysis, and Economic Policies Book Series,in: Rodrigo Alfaro (ed.), Financial Stability, Monetary Policy, and Central Banking, edition 1, volume 15, chapter 12, pages 371-410 Central Bank of Chile.
    4. Detken, Carsten & Alessi, Lucia, 2014. "Identifying excessive credit growth and leverage," Working Paper Series 1723, European Central Bank.
    5. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    6. Andrew K. Rose & Mark M. Spiegel, "undated". "Cross-Country Causes and Consequences of the 2008 Crisis: Early Warning," Working Papers 6, Department of the Treasury, Ministry of the Economy and of Finance.
    7. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency Crashes in Emerging Markets: Empirical Indicators," Center for International and Development Economics Research (CIDER) Working Papers 233424, University of California-Berkeley, Department of Economics.
    8. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    9. Krugman, Paul, 1979. "A Model of Balance-of-Payments Crises," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 11(3), pages 311-325, August.
    10. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    11. Asli Demirgüç-Kunt & Enrica Detragiache, 2005. "Cross-Country Empirical Studies of Systemic Bank Distress: A Survey," National Institute Economic Review, National Institute of Economic and Social Research, vol. 192(1), pages 68-83, April.
    12. repec:erf:erfstu:78 is not listed on IDEAS
    13. Helmut Elsinger & Alfred Lehar & Martin Summer, 2006. "Using Market Information for Banking System Risk Assessment," International Journal of Central Banking, International Journal of Central Banking, vol. 2(1), March.
    14. Davis, E. Philip & Karim, Dilruba, 2008. "Comparing early warning systems for banking crises," Journal of Financial Stability, Elsevier, vol. 4(2), pages 89-120, June.
    15. Abdul d Abiad, 2003. "Early Warning Systems; A Survey and a Regime-Switching Approach," IMF Working Papers 03/32, International Monetary Fund.
    16. Forbes, Kristin J. & Warnock, Francis E., 2012. "Capital flow waves: Surges, stops, flight, and retrenchment," Journal of International Economics, Elsevier, vol. 88(2), pages 235-251.
    17. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency crashes in emerging markets: An empirical treatment," Journal of International Economics, Elsevier, vol. 41(3-4), pages 351-366, November.
    18. Lilien, David M, 1982. "Sectoral Shifts and Cyclical Unemployment," Journal of Political Economy, University of Chicago Press, vol. 90(4), pages 777-793, August.
    19. António R. Antunes & Diana Bonfim & Nuno Monteiro & Paulo M.M. Rodrigues, 2014. "Early Warning Indicators of Banking Crises: Exploring new Data and Tools," Economic Bulletin and Financial Stability Report Articles, Banco de Portugal, Economics and Research Department.
    20. Fabian Valencia & Luc Laeven, 2008. "Systemic Banking Crises; A New Database," IMF Working Papers 08/224, International Monetary Fund.
    21. Alejandro Gaytán & Christian A. Johnson, 2002. "A Review of the Literature on Early Warning Systems for Banking Crises," Working Papers Central Bank of Chile 183, Central Bank of Chile.
    22. Luc Laeven & Fabián Valencia, 2013. "Systemic Banking Crises Database," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 61(2), pages 225-270, June.
    23. Jeffrey A. Frankel & George Saravelos, 2010. "Are Leading Indicators of Financial Crises Useful for Assessing Country Vulnerability? Evidence from the 2008-09 Global Crisis," NBER Working Papers 16047, National Bureau of Economic Research, Inc.
    24. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. repec:eee:finlet:v:21:y:2017:i:c:p:277-283 is not listed on IDEAS
    2. Berlinger, Edina, 2017. "Implicit rating: A potential new method to alert crisis on the interbank lending market," Finance Research Letters, Elsevier, vol. 21(C), pages 277-283.
    3. Lukasz, Rachel & Fisher, Jack, 2016. "Assessing vulnerabilities to financial shocks in some key global economies," Bank of England working papers 636, Bank of England.
    4. Mathias Drehmann & Anamaria Illes & Mikael Juselius & Marjorie Santos, 2015. "How much income is used for debt payments? A new database for debt service ratios," BIS Quarterly Review, Bank for International Settlements, September.
    5. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    6. Ons Jedidi & Jean-Sébastien Pentecôte, 2015. "Prédire les crises bancaires : un système d’alerte robuste," Revue française d'économie, Presses de Sciences-Po, vol. 0(3), pages 189-225.
    7. Caggiano, Giovanni & Calice, Pietro & Leonida, Leone & Kapetanios, George, 2016. "Comparing logit-based early warning systems: Does the duration of systemic banking crises matter?," Journal of Empirical Finance, Elsevier, vol. 37(C), pages 104-116.
    8. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682,, revised Apr 2016.
    9. Domonkos Tomáš & Ostrihoň Filip & Šikulová Ivana & Širaňová Mária, 2017. "Analysing the Relevance of the MIP Scoreboard's Indicators," National Institute Economic Review, National Institute of Economic and Social Research, vol. 239(1), pages 32-52, February.

    More about this item


    Early warning systems; financial crisis; Bayesian model averaging;

    JEL classification:

    • G01 - Financial Economics - - General - - - Financial Crises

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:62194. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Joachim Winter). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.