IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20202408.html
   My bibliography  Save this paper

Random forest versus logit models: which offers better early warning of fiscal stress?

Author

Listed:
  • Jarmulska, Barbara

Abstract

This study seeks to answer whether it is possible to design an early warning system framework that can signal the risk of fiscal stress in the near future, and what shape such a system should take. To do so, multiple models based on econometric logit and the random forest models are designed and compared. Using a dataset of 20 annual frequency variables pertaining to 43 advanced and emerging countries during 1992-2018, the results confirm the possibility of obtaining an effective model, which correctly predicts 70-80% of fiscal stress events and tranquil periods. The random forest-based early warning model outperforms logit models. While the random forest model is commonly understood to provide lower interpretability than logit models do, this study employs tools that can be used to provide useful information for understanding what is behind the black-box. These tools can provide information on the most important explanatory variables and on the shape of the relationship between these variables and the outcome classification. Thus, the study contributes to the discussion on the usefulness of machine learning methods in economics. JEL Classification: C40, C53, H63, G01

Suggested Citation

  • Jarmulska, Barbara, 2020. "Random forest versus logit models: which offers better early warning of fiscal stress?," Working Paper Series 2408, European Central Bank.
  • Handle: RePEc:ecb:ecbwps:20202408
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecb.wp2408~aa6b05aed7.en.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    2. Kumar, Mohan & Moorthy, Uma & Perraudin, William, 2003. "Predicting emerging market currency crashes," Journal of Empirical Finance, Elsevier, vol. 10(4), pages 427-454, September.
    3. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    4. Ciarlone, Alessio & Trebeschi, Giorgio, 2005. "Designing an early warning system for debt crises," Emerging Markets Review, Elsevier, vol. 6(4), pages 376-395, December.
    5. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    6. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Is the 2007 U.S. Sub-Prime Financial Crisis So Different? An International Historical Comparison," Panoeconomicus, Savez ekonomista Vojvodine, Novi Sad, Serbia, vol. 56(3), pages 291-299, September.
    7. Moritz Schularick & Alan M. Taylor, 2012. "Credit Booms Gone Bust: Monetary Policy, Leverage Cycles, and Financial Crises, 1870-2008," American Economic Review, American Economic Association, vol. 102(2), pages 1029-1061, April.
    8. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    9. Fioramanti, Marco, 2008. "Predicting sovereign debt crises using artificial neural networks: A comparative approach," Journal of Financial Stability, Elsevier, vol. 4(2), pages 149-164, June.
    10. Manasse, Paolo & Roubini, Nouriel, 2009. ""Rules of thumb" for sovereign debt crises," Journal of International Economics, Elsevier, vol. 78(2), pages 192-205, July.
    11. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    12. Mr. Paul Cashin & Rupa Duttagupta, 2008. "The Anatomy of Banking Crises," IMF Working Papers 2008/093, International Monetary Fund.
    13. Benedict S. Jimenez, 2017. "Institutional Constraints, Rule-Following, and Circumvention: Tax and Expenditure Limits and the Choice of Fiscal Tools During a Budget Crisis," Public Budgeting & Finance, Wiley Blackwell, vol. 37(2), pages 5-34, June.
    14. Bluwstein, Kristina & Buckmann, Marcus & Joseph, Andreas & Kang, Miao & Kapadia, Sujit & Simsek, Özgür, 2020. "Credit growth, the yield curve and financial crisis prediction: evidence from a machine learning approach," Bank of England working papers 848, Bank of England.
    15. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    16. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    17. Carmen M. Reinhart & Kenneth S. Rogoff, 2011. "From Financial Crash to Debt Crisis," American Economic Review, American Economic Association, vol. 101(5), pages 1676-1706, August.
    18. Paolo Manasse & Mr. Axel Schimmelpfennig & Nouriel Roubini, 2003. "Predicting Sovereign Debt Crises," IMF Working Papers 2003/221, International Monetary Fund.
    19. Demirgüç-Kunt, Asli & Detragiache, Enrica, 2005. "Cross-Country Empirical Studies of Systemic Bank Distress: A Survey," National Institute Economic Review, Cambridge University Press, vol. 192, pages 68-83, April.
    20. Demyanyk, Yuliya & Hasan, Iftekhar, 2010. "Financial crises and bank failures: A review of prediction methods," Omega, Elsevier, vol. 38(5), pages 315-324, October.
    21. Fuertes, Ana-Maria & Kalotychou, Elena, 2006. "Early warning systems for sovereign debt crises: The role of heterogeneity," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1420-1441, November.
    22. Fuertes, Ana-Maria & Kalotychou, Elena, 2007. "Optimal design of early warning systems for sovereign debt crises," International Journal of Forecasting, Elsevier, vol. 23(1), pages 85-100.
    23. Juan J. Cruces & Christoph Trebesch, 2013. "Sovereign Defaults: The Price of Haircuts," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(3), pages 85-117, July.
    24. Markus Holopainen & Peter Sarlin, 2017. "Toward robust early-warning models: a horse race, ensembles and model uncertainty," Quantitative Finance, Taylor & Francis Journals, vol. 17(12), pages 1933-1963, December.
    25. David Beers & Jean-Sébastien Nadeau, 2014. "Database of Sovereign Defaults, 2015 (Revised May 2015)," Technical Reports 101, Bank of Canada.
    26. Iva Petrova & Nazim Belhocine & Miss Gabriela Dobrescu & Mr. Emanuele Baldacci, 2011. "Assessing Fiscal Stress," IMF Working Papers 2011/100, International Monetary Fund.
    27. Lo Luca, Marco & Peltonen, Tuomas, 2011. "Macro-financial vulnerabilities and future financial stress : Assessing systemic risks and predicting systemic events," BOFIT Discussion Papers 2/2011, Bank of Finland, Institute for Economies in Transition.
    28. David T. Mitchell & Dean Stansel, 2016. "The Determinants of the Severity of State Fiscal Crises," Public Budgeting & Finance, Wiley Blackwell, vol. 36(4), pages 50-67, December.
    29. Barrell, Ray & Davis, E. Philip & Karim, Dilruba & Liadze, Iana, 2010. "Bank regulation, property prices and early warning systems for banking crises in OECD countries," Journal of Banking & Finance, Elsevier, vol. 34(9), pages 2255-2264, September.
    30. E. Philip Davis & Dilruba Karim, 2008. "Could Early Warning Systems Have Helped To Predict the Sub-Prime Crisis?," National Institute Economic Review, National Institute of Economic and Social Research, vol. 206(1), pages 35-47, October.
    31. Pablo Hernández de Cos & Enrique Moral-Benito & Gerrit B. Koester & Christiane Nickel, 2014. "Signalling fiscal stress in the euro area: A country-specific early warning system," Working Papers 1418, Banco de España.
    32. Marco Lo Duca & Tuomas Peltonen, 2011. "Macrofinancial vulnerabilities and future financial stress: assessing systemic risks and predicting systemic events," BIS Papers chapters, in: Bank for International Settlements (ed.), Macroprudential regulation and policy, volume 60, pages 82-88, Bank for International Settlements.
    33. Katia Berti & Matteo Salto & Matthieu Lequien, 2012. "An early-detection index of fiscal stress for EU countries," European Economy - Economic Papers 2008 - 2015 475, Directorate General Economic and Financial Affairs (DG ECFIN), European Commission.
    34. Christoph Trebesch & Mr. Udaibir S Das & Mr. Michael G. Papaioannou, 2012. "Sovereign Debt Restructurings 1950-2010: Literature Survey, Data, and Stylized Facts," IMF Working Papers 2012/203, International Monetary Fund.
    35. Graciela Laura Kaminsky, 1999. "Currency and Banking Crises: The Early Warnings of Distress," IMF Working Papers 1999/178, International Monetary Fund.
    36. Kar‐yiu Wong & Richard Y. K. Ho, 2002. "The Asian Crisis, 1997," Review of International Economics, Wiley Blackwell, vol. 10(1), pages 1-1, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lanbiao Liu & Chen Chen & Bo Wang, 2022. "Predicting financial crises with machine learning methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(5), pages 871-910, August.
    2. Tamás Kristóf, 2021. "Sovereign Default Forecasting in the Era of the COVID-19 Crisis," JRFM, MDPI, vol. 14(10), pages 1-24, October.
    3. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    4. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    5. Fendel Ralf & Stremmel Hanno, 2016. "Characteristics of Banking Crises: A Comparative Study with Geographical Contagion," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(3), pages 349-388, May.
    6. Stijn Claessens & M. Ayhan Kose, 2013. "Financial Crises: Explanations, Types and Implications," CAMA Working Papers 2013-06, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    7. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.
    8. Betz, Frank & Oprică, Silviu & Peltonen, Tuomas A. & Sarlin, Peter, 2014. "Predicting distress in European banks," Journal of Banking & Finance, Elsevier, vol. 45(C), pages 225-241.
    9. Tran Huynh & Silke Uebelmesser, 2022. "Early warning models for systemic banking crises: can political indicators improve prediction?," Jena Economic Research Papers 2022-007, Friedrich-Schiller-University Jena.
    10. Mr. Jiro Honda & Rene Tapsoba & Ismael Issifou, 2018. "When Do We Repair the Roof? Insights from Responses to Fiscal Crisis Early Warning Signals," IMF Working Papers 2018/077, International Monetary Fund.
    11. Tölö, Eero, 2020. "Predicting systemic financial crises with recurrent neural networks," Journal of Financial Stability, Elsevier, vol. 49(C).
    12. Mr. Paulo A Medas & Pranav Gupta & Ms. Marialuz Moreno Badia & Yuan Xiang, 2020. "Debt Is Not Free," IMF Working Papers 2020/001, International Monetary Fund.
    13. Rho, Caterina & Saenz, Manrique, 2021. "Financial stress and the probability of sovereign default," Journal of International Money and Finance, Elsevier, vol. 110(C).
    14. Mathonnat, Clément & Minea, Alexandru, 2018. "Financial development and the occurrence of banking crises," Journal of Banking & Finance, Elsevier, vol. 96(C), pages 344-354.
    15. Filippopoulou, Chryssanthi & Galariotis, Emilios & Spyrou, Spyros, 2020. "An early warning system for predicting systemic banking crises in the Eurozone: A logit regression approach," Journal of Economic Behavior & Organization, Elsevier, vol. 172(C), pages 344-363.
    16. Hamdaoui, Mekki, 2016. "Are systemic banking crises in developed and developing countries predictable?," Journal of Multinational Financial Management, Elsevier, vol. 37, pages 114-138.
    17. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
    18. Lainà, Patrizio & Nyholm, Juho & Sarlin, Peter, 2015. "Leading indicators of systemic banking crises: Finland in a panel of EU countries," Review of Financial Economics, Elsevier, vol. 24(C), pages 18-35.
    19. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    20. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2018. "An evaluation of early warning models for systemic banking crises: Does machine learning improve predictions?," Discussion Papers 48/2018, Deutsche Bundesbank.

    More about this item

    Keywords

    early warning system; interpretability of machine learning; predictive performance;
    All these keywords.

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • H63 - Public Economics - - National Budget, Deficit, and Debt - - - Debt; Debt Management; Sovereign Debt
    • G01 - Financial Economics - - General - - - Financial Crises

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20202408. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/emieude.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.