IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1501.04682.html
   My bibliography  Save this paper

Toward robust early-warning models: A horse race, ensembles and model uncertainty

Author

Listed:
  • Markus Holopainen
  • Peter Sarlin

Abstract

This paper presents first steps toward robust models for crisis prediction. We conduct a horse race of conventional statistical methods and more recent machine learning methods as early-warning models. As individual models are in the literature most often built in isolation of other methods, the exercise is of high relevance for assessing the relative performance of a wide variety of methods. Further, we test various ensemble approaches to aggregating the information products of the built models, providing a more robust basis for measuring country-level vulnerabilities. Finally, we provide approaches to estimating model uncertainty in early-warning exercises, particularly model performance uncertainty and model output uncertainty. The approaches put forward in this paper are shown with Europe as a playground. Generally, our results show that the conventional statistical approaches are outperformed by more advanced machine learning methods, such as k-nearest neighbors and neural networks, and particularly by model aggregation approaches through ensemble learning.

Suggested Citation

  • Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
  • Handle: RePEc:arx:papers:1501.04682
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1501.04682
    File Function: Latest version
    Download Restriction: no

    References listed on IDEAS

    as
    1. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Varieties of Crises and Their Dates," Introductory Chapters,in: This Time Is Different: Eight Centuries of Financial Folly Princeton University Press.
    2. Carmen M. Reinhart & Kenneth S. Rogoff, 2014. "This Time is Different: A Panoramic View of Eight Centuries of Financial Crises," Annals of Economics and Finance, Society for AEF, vol. 15(2), pages 1065-1188, November.
    3. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    4. repec:taf:jnlbes:v:35:y:2017:i:4:p:499-512 is not listed on IDEAS
    5. Lainà, Patrizio & Nyholm, Juho & Sarlin, Peter, 2015. "Leading indicators of systemic banking crises: Finland in a panel of EU countries," Review of Financial Economics, Elsevier, vol. 24(C), pages 18-35.
    6. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    7. Andrew Berg & Eduardo Borensztein & Catherine Pattillo, 2005. "Assessing Early Warning Systems: How Have They Worked in Practice?," IMF Staff Papers, Palgrave Macmillan, vol. 52(3), pages 1-5.
    8. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency crashes in emerging markets: An empirical treatment," Journal of International Economics, Elsevier, vol. 41(3-4), pages 351-366, November.
    9. El-Shagi, M. & Knedlik, T. & von Schweinitz, G., 2013. "Predicting financial crises: The (statistical) significance of the signals approach," Journal of International Money and Finance, Elsevier, vol. 35(C), pages 76-103.
    10. Frankel, Jeffrey A. & Rose, Andrew K., 1996. "Currency Crashes in Emerging Markets: Empirical Indicators," Center for International and Development Economics Research (CIDER) Working Papers 233424, University of California-Berkeley, Department of Economics.
    11. Edward I. Altman, 1968. "Financial Ratios, Discriminant Analysis And The Prediction Of Corporate Bankruptcy," Journal of Finance, American Finance Association, vol. 23(4), pages 589-609, September.
    12. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    13. Peter Sarlin, 2014. "Macroprudential oversight, risk communication and visualization," Papers 1404.4550, arXiv.org, revised Jun 2014.
    14. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    15. Detken, Carsten & Alessi, Lucia, 2014. "Identifying excessive credit growth and leverage," Working Paper Series 1723, European Central Bank.
    16. Bussiere, Matthieu & Fratzscher, Marcel, 2006. "Towards a new early warning system of financial crises," Journal of International Money and Finance, Elsevier, vol. 25(6), pages 953-973, October.
    17. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    18. Schmidt, Reinhart, 1984. "Early warning of debt rescheduling," Journal of Banking & Finance, Elsevier, vol. 8(2), pages 357-370, June.
    19. Christophe Hurlin & Sébastien Laurent & Rogier Quaedvlieg & Stephan Smeekes, 2017. "Risk Measure Inference," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(4), pages 499-512, October.
    20. Marcos Chamon & Paolo Manasse & Alessandro Prati, 2007. "Can We Predict the Next Capital Account Crisis?," IMF Staff Papers, Palgrave Macmillan, vol. 54(2), pages 270-305, June.
    21. Barrell, Ray & Davis, E. Philip & Karim, Dilruba & Liadze, Iana, 2010. "Bank regulation, property prices and early warning systems for banking crises in OECD countries," Journal of Banking & Finance, Elsevier, vol. 34(9), pages 2255-2264, September.
    22. Barry Eichengreen & Andrew K. Rose, 1998. "Staying Afloat When the Wind Shifts: External Factors and Emerging-Market Banking Crises," NBER Working Papers 6370, National Bureau of Economic Research, Inc.
    23. Demyanyk, Yuliya & Hasan, Iftekhar, 2010. "Financial crises and bank failures: A review of prediction methods," Omega, Elsevier, vol. 38(5), pages 315-324, October.
    24. Sarlin, Peter, 2013. "On policymakers’ loss functions and the evaluation of early warning systems," Economics Letters, Elsevier, vol. 119(1), pages 1-7.
    25. Frank, Charles Jr. & Cline, William R., 1971. "Measurement of debt servicing capacity: An application of discriminant analysis," Journal of International Economics, Elsevier, vol. 1(3), pages 327-344, August.
    26. Sarlin, Peter & Peltonen, Tuomas A., 2013. "Mapping the state of financial stability," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 26(C), pages 46-76.
    27. Fioramanti, Marco, 2008. "Predicting sovereign debt crises using artificial neural networks: A comparative approach," Journal of Financial Stability, Elsevier, vol. 4(2), pages 149-164, June.
    28. André Fourçans & Raphaël Franck, 2003. "Currency Crises," Books, Edward Elgar Publishing, number 3124.
    29. Sarlin, Peter & Peltonen, Tuomas A., 2013. "Mapping the state of financial stability," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 26(C), pages 46-76.
    30. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2011. "Anchoring Countercyclical Capital Buffers: The role of Credit Aggregates," International Journal of Central Banking, International Journal of Central Banking, vol. 7(4), pages 189-240, December.
    31. Berg, Andrew & Pattillo, Catherine, 1999. "Predicting currency crises:: The indicators approach and an alternative," Journal of International Money and Finance, Elsevier, vol. 18(4), pages 561-586, August.
    32. Peltonen, Tuomas, 2006. "Are emerging market currency crises predictable? A test," Working Paper Series 571, European Central Bank.
    33. Alessi, Lucia & Antunes, Antonio & Babecky, Jan & Baltussen, Simon & Behn, Markus & Bonfim, Diana & Bush, Oliver & Detken, Carsten & Frost, Jon & Guimaraes, Rodrigo & Havranek, Tomas & Joy, Mark & Kau, 2015. "Comparing different early warning systems: Results from a horse race competition among members of the Macro-prudential Research Network," MPRA Paper 62194, University Library of Munich, Germany.
    34. Duttagupta, Rupa & Cashin, Paul, 2011. "Anatomy of banking crises in developing and emerging market countries," Journal of International Money and Finance, Elsevier, vol. 30(2), pages 354-376, March.
    35. repec:bla:joares:v:4:y:1966:i::p:71-111 is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carlos León & José Fernando Moreno & Jorge Cely, 2016. "Whose Balance Sheet is this? Neural Networks for Banks’ Pattern Recognition," Borradores de Economia 959, Banco de la Republica de Colombia.
    2. repec:ijc:ijcjou:y:2018:q:1:a:2 is not listed on IDEAS
    3. Oet, Mikhail V. & Dooley, John & Gramlich, Dieter & Sarlin, Peter & Ong, Stephen J., 2015. "Evaluating the Information Value for Measures of Systemic Conditions," Working Paper 1513, Federal Reserve Bank of Cleveland.
    4. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    5. Tjeerd M. Boonman & Jan P. A. M. Jacobs & Gerard H. Kuper & Alberto Romero, 2017. "Early Warning Systems for Currency Crises with Real-Time Data," CIRANO Working Papers 2017s-18, CIRANO.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1501.04682. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (arXiv administrators). General contact details of provider: http://arxiv.org/ .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.