IDEAS home Printed from https://ideas.repec.org/a/eee/finsta/v45y2019ics1572308918305801.html
   My bibliography  Save this article

Does machine learning help us predict banking crises?

Author

Listed:
  • Beutel, Johannes
  • List, Sophia
  • von Schweinitz, Gregor

Abstract

This paper compares the out-of-sample predictive performance of different early warning models for systemic banking crises using a sample of advanced economies covering the past 45 years. We compare a benchmark logit approach to several machine learning approaches recently proposed in the literature. We find that while machine learning methods often attain a very high in-sample fit, they are outperformed by the logit approach in recursive out-of-sample evaluations. This result is robust to the choice of performance metric, crisis definition, preference parameter, and sample length, as well as to using different sets of variables and data transformations. Thus, our paper suggests that further enhancements to machine learning early warning models are needed before they are able to offer a substantial value-added for predicting systemic banking crises. Conventional logit models appear to use the available information already fairly efficiently, and would for instance have been able to predict the 2007/2008 financial crisis out-of-sample for many countries. In line with economic intuition, these models identify credit expansions, asset price booms and external imbalances as key predictors of systemic banking crises.

Suggested Citation

  • Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
  • Handle: RePEc:eee:finsta:v:45:y:2019:i:c:s1572308918305801
    DOI: 10.1016/j.jfs.2019.100693
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1572308918305801
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.jfs.2019.100693?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    2. McFadden, Daniel L., 1984. "Econometric analysis of qualitative response models," Handbook of Econometrics, in: Z. Griliches† & M. D. Intriligator (ed.), Handbook of Econometrics, edition 1, volume 2, chapter 24, pages 1395-1457, Elsevier.
    3. Luc Laeven & Fabian Valencia, 2020. "Systemic Banking Crises Database II," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 68(2), pages 307-361, June.
    4. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    5. Schüler, Yves Stephan & Hiebert, Paul P. & Peltonen, Tuomas A., 2015. "Characterising the financial cycle: A multivariate and time-varying approach," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 112985, Verein für Socialpolitik / German Economic Association.
    6. Carmen M. Reinhart & Kenneth S. Rogoff, 2014. "This Time is Different: A Panoramic View of Eight Centuries of Financial Crises," Annals of Economics and Finance, Society for AEF, vol. 15(2), pages 215-268, November.
    7. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency Crashes in Emerging Markets: Empirical Indicators," Center for International and Development Economics Research (CIDER) Working Papers 233424, University of California-Berkeley, Department of Economics.
    8. Sarlin, Peter, 2013. "On policymakers’ loss functions and the evaluation of early warning systems," Economics Letters, Elsevier, vol. 119(1), pages 1-7.
    9. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    10. Sarlin, Peter & von Schweinitz, Gregor, 2021. "Optimizing Policymakers’ Loss Functions In Crisis Prediction: Before, Within Or After?," Macroeconomic Dynamics, Cambridge University Press, vol. 25(1), pages 100-123, January.
    11. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    12. Kaminsky, Graciela L. & Reinhart, Carmen M., 2000. "On crises, contagion, and confusion," Journal of International Economics, Elsevier, vol. 51(1), pages 145-168, June.
    13. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    14. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Varieties of Crises and Their Dates," Introductory Chapters, in: This Time Is Different: Eight Centuries of Financial Folly, Princeton University Press.
    15. Katsuyuki Tanaka & Takuji Kinkyo & Shigeyuki Hamori, 2018. "Financial Hazard Map: Financial Vulnerability Predicted by a Random Forests Classification Model," Sustainability, MDPI, vol. 10(5), pages 1-18, May.
    16. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    17. Diebold, Francis X & Rudebusch, Glenn D, 1989. "Scoring the Leading Indicators," The Journal of Business, University of Chicago Press, vol. 62(3), pages 369-391, July.
    18. Òscar Jordà & Moritz Schularick & Alan M Taylor, 2011. "Financial Crises, Credit Booms, and External Imbalances: 140 Years of Lessons," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 59(2), pages 340-378, June.
    19. Markus K. Brunnermeier, 2009. "Deciphering the Liquidity and Credit Crunch 2007-2008," Journal of Economic Perspectives, American Economic Association, vol. 23(1), pages 77-100, Winter.
    20. Makram El-Shagi & Axel Lindner & Gregor von Schweinitz, 2016. "Real Effective Exchange Rate Misalignment in the Euro Area: A Counterfactual Analysis," Review of International Economics, Wiley Blackwell, vol. 24(1), pages 37-66, February.
    21. Atsushi Inoue & Lutz Kilian, 2005. "In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?," Econometric Reviews, Taylor & Francis Journals, vol. 23(4), pages 371-402.
    22. Cerutti, Eugenio & Claessens, Stijn & Laeven, Luc, 2017. "The use and effectiveness of macroprudential policies: New evidence," Journal of Financial Stability, Elsevier, vol. 28(C), pages 203-224.
    23. Lo Duca, Marco & Koban, Anne & Basten, Marisa & Bengtsson, Elias & Klaus, Benjamin & Kusmierczyk, Piotr & Lang, Jan Hannes & Detken, Carsten & Peltonen, Tuomas, 2017. "A new database for financial crises in European countries," Occasional Paper Series 194, European Central Bank.
    24. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2012. "Characterising the financial cycle: don't lose sight of the medium term!," BIS Working Papers 380, Bank for International Settlements.
    25. Russell Davidson & James MacKinnon, 2000. "Bootstrap tests: how many bootstraps?," Econometric Reviews, Taylor & Francis Journals, vol. 19(1), pages 55-68.
    26. Frédéric Boissay & Fabrice Collard & Frank Smets, 2016. "Booms and Banking Crises," Journal of Political Economy, University of Chicago Press, vol. 124(2), pages 489-538.
    27. Stijn Claessens, 2015. "An Overview of Macroprudential Policy Tools," Annual Review of Financial Economics, Annual Reviews, vol. 7(1), pages 397-422, December.
    28. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Is the 2007 US Sub-Prime Financial Crisis So Different?: An International Historical Comparison," Panoeconomicus, Savez ekonomista Vojvodine, Novi Sad, Serbia, vol. 56(3), pages 291-299.
    29. Raghuram G. Rajan, 2006. "Has Finance Made the World Riskier?," European Financial Management, European Financial Management Association, vol. 12(4), pages 499-533, September.
    30. Guillermo A. Calvo, 1998. "Capital Flows and Capital-Market Crises: The Simple Economics of Sudden Stops," Journal of Applied Economics, Universidad del CEMA, vol. 1, pages 35-54, November.
    31. Pierre-Olivier Gourinchas & Maurice Obstfeld, 2012. "Stories of the Twentieth Century for the Twenty-First," American Economic Journal: Macroeconomics, American Economic Association, vol. 4(1), pages 226-265, January.
    32. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2011. "Anchoring Countercyclical Capital Buffers: The role of Credit Aggregates," International Journal of Central Banking, International Journal of Central Banking, vol. 7(4), pages 189-240, December.
    33. Drehmann, Mathias & Juselius, Mikael & Korinek, Anton, 2017. "Accounting for debt service: The painful legacy of credit booms," Bank of Finland Research Discussion Papers 12/2017, Bank of Finland.
    34. Duprey, Thibaut & Klaus, Benjamin & Peltonen, Tuomas, 2017. "Dating systemic financial stress episodes in the EU countries," Journal of Financial Stability, Elsevier, vol. 32(C), pages 30-56.
    35. Charles P. Kindleberger & Robert Z. Aliber, 2005. "Manias, Panics and Crashes," Palgrave Macmillan Books, Palgrave Macmillan, edition 0, number 978-0-230-62804-5.
    36. El-Shagi, M. & Knedlik, T. & von Schweinitz, G., 2013. "Predicting financial crises: The (statistical) significance of the signals approach," Journal of International Money and Finance, Elsevier, vol. 35(C), pages 76-103.
    37. A. Berg & C. Pattillo, 1999. "What Caused the Asian Crises: An Early Warning System Approach," Economic Notes, Banca Monte dei Paschi di Siena SpA, vol. 28(3), pages 285-334, November.
    38. Brunnermeier, Markus K. & Oehmke, Martin, 2013. "Bubbles, Financial Crises, and Systemic Risk," Handbook of the Economics of Finance, in: G.M. Constantinides & M. Harris & R. M. Stulz (ed.), Handbook of the Economics of Finance, volume 2, chapter 0, pages 1221-1288, Elsevier.
    39. repec:zbw:bofrdp:2017_012 is not listed on IDEAS
    40. International Monetary Fund, 2011. "Macroprudential Policy: What Instruments and How to Use them? Lessons From Country Experiences," IMF Working Papers 2011/238, International Monetary Fund.
    41. Markus Holopainen & Peter Sarlin, 2017. "Toward robust early-warning models: a horse race, ensembles and model uncertainty," Quantitative Finance, Taylor & Francis Journals, vol. 17(12), pages 1933-1963, December.
    42. Angela Maddaloni & Jose-Luis Peydro, 2011. "Bank Risk-taking, Securitization, Supervision, and Low Interest Rates: Evidence from the Euro-area and the U.S. Lending Standards," The Review of Financial Studies, Society for Financial Studies, vol. 24(6), pages 2121-2165.
    43. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency crashes in emerging markets: An empirical treatment," Journal of International Economics, Elsevier, vol. 41(3-4), pages 351-366, November.
    44. Bussiere, Matthieu & Fratzscher, Marcel, 2006. "Towards a new early warning system of financial crises," Journal of International Money and Finance, Elsevier, vol. 25(6), pages 953-973, October.
    45. Tanaka, Katsuyuki & Kinkyo, Takuji & Hamori, Shigeyuki, 2016. "Random forests-based early warning system for bank failures," Economics Letters, Elsevier, vol. 148(C), pages 118-121.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2018. "An evaluation of early warning models for systemic banking crises: Does machine learning improve predictions?," Discussion Papers 48/2018, Deutsche Bundesbank.
    2. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    3. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    4. Xianglong Liu, 2023. "Towards Better Banking Crisis Prediction: Could an Automatic Variable Selection Process Improve the Performance?," The Economic Record, The Economic Society of Australia, vol. 99(325), pages 288-312, June.
    5. Tran Huynh & Silke Uebelmesser, 2022. "Early warning models for systemic banking crises: can political indicators improve prediction?," Jena Economics Research Papers 2022-007, Friedrich-Schiller-University Jena.
    6. Lo Duca, Marco & Koban, Anne & Basten, Marisa & Bengtsson, Elias & Klaus, Benjamin & Kusmierczyk, Piotr & Lang, Jan Hannes & Detken, Carsten & Peltonen, Tuomas, 2017. "A new database for financial crises in European countries," ESRB Occasional Paper Series 13, European Systemic Risk Board.
    7. Dieckelmann, Daniel, 2020. "Cross-border lending and the international transmission of banking crises," Discussion Papers 2020/13, Free University Berlin, School of Business & Economics.
    8. Casabianca, Elizabeth Jane & Catalano, Michele & Forni, Lorenzo & Giarda, Elena & Passeri, Simone, 2022. "A machine learning approach to rank the determinants of banking crises over time and across countries," Journal of International Money and Finance, Elsevier, vol. 129(C).
    9. Bordo, M.D. & Meissner, C.M., 2016. "Fiscal and Financial Crises," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 355-412, Elsevier.
    10. Sarlin, Peter & von Schweinitz, Gregor, 2021. "Optimizing Policymakers’ Loss Functions In Crisis Prediction: Before, Within Or After?," Macroeconomic Dynamics, Cambridge University Press, vol. 25(1), pages 100-123, January.
    11. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    12. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    13. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    14. Marcin Pietrzak, 2021. "Can Financial Soundness Indicators Help Predict Financial Sector Distress?," IMF Working Papers 2021/197, International Monetary Fund.
    15. Duprey, Thibaut & Klaus, Benjamin, 2022. "Early warning or too late? A (pseudo-)real-time identification of leading indicators of financial stress," Journal of Banking & Finance, Elsevier, vol. 138(C).
    16. Roy, Saktinil & Kemme, David M., 2012. "Causes of banking crises: Deregulation, credit booms and asset bubbles, then and now," International Review of Economics & Finance, Elsevier, vol. 24(C), pages 270-294.
    17. Cesa-Bianchi, Ambrogio & Eguren Martin, Fernando & Thwaites, Gregory, 2019. "Foreign booms, domestic busts: The global dimension of banking crises," Journal of Financial Intermediation, Elsevier, vol. 37(C), pages 58-74.
    18. Catão, Luis A.V. & Milesi-Ferretti, Gian Maria, 2014. "External liabilities and crises," Journal of International Economics, Elsevier, vol. 94(1), pages 18-32.
    19. policy, Work stream on macroprudential & Albertazzi, Ugo & Martin, Alberto & Assouan, Emmanuelle & Tristani, Oreste & Galati, Gabriele & Vlassopoulos, Thomas, 2021. "The role of financial stability considerations in monetary policy and the interaction with macroprudential policy in the euro area," Occasional Paper Series 272, European Central Bank.
    20. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).

    More about this item

    Keywords

    Early warning system; Logit; Machine learning; Systemic banking crises;
    All these keywords.

    JEL classification:

    • C35 - Mathematical and Quantitative Methods - - Multiple or Simultaneous Equation Models; Multiple Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • G01 - Financial Economics - - General - - - Financial Crises

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finsta:v:45:y:2019:i:c:s1572308918305801. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jfstabil .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.