IDEAS home Printed from https://ideas.repec.org/p/ecb/ecbwps/20202408.html
   My bibliography  Save this paper

Random forest versus logit models: which offers better early warning of fiscal stress?

Author

Listed:
  • Jarmulska, Barbara

Abstract

This study seeks to answer whether it is possible to design an early warning system framework that can signal the risk of fiscal stress in the near future, and what shape such a system should take. To do so, multiple models based on econometric logit and the random forest models are designed and compared. Using a dataset of 20 annual frequency variables pertaining to 43 advanced and emerging countries during 1992-2018, the results confirm the possibility of obtaining an effective model, which correctly predicts 70-80% of fiscal stress events and tranquil periods. The random forest-based early warning model outperforms logit models. While the random forest model is commonly understood to provide lower interpretability than logit models do, this study employs tools that can be used to provide useful information for understanding what is behind the black-box. These tools can provide information on the most important explanatory variables and on the shape of the relationship between these variables and the outcome classification. Thus, the study contributes to the discussion on the usefulness of machine learning methods in economics. JEL Classification: C40, C53, H63, G01

Suggested Citation

  • Jarmulska, Barbara, 2020. "Random forest versus logit models: which offers better early warning of fiscal stress?," Working Paper Series 2408, European Central Bank.
  • Handle: RePEc:ecb:ecbwps:20202408
    as

    Download full text from publisher

    File URL: https://www.ecb.europa.eu//pub/pdf/scpwps/ecb.wp2408~aa6b05aed7.en.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mr. Emanuele Baldacci & Iva Petrova & Nazim Belhocine & Miss Gabriela Dobrescu, 2011. "Assessing Fiscal Stress," IMF Working Papers 2011/100, International Monetary Fund.
    2. Lo Luca, Marco & Peltonen, Tuomas, 2011. "Macro-financial vulnerabilities and future financial stress : Assessing systemic risks and predicting systemic events," BOFIT Discussion Papers 2/2011, Bank of Finland, Institute for Economies in Transition.
    3. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    4. David T. Mitchell & Dean Stansel, 2016. "The Determinants of the Severity of State Fiscal Crises," Public Budgeting & Finance, Wiley Blackwell, vol. 36(4), pages 50-67, December.
    5. Kumar, Mohan & Moorthy, Uma & Perraudin, William, 2003. "Predicting emerging market currency crashes," Journal of Empirical Finance, Elsevier, vol. 10(4), pages 427-454, September.
    6. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    7. Ciarlone, Alessio & Trebeschi, Giorgio, 2005. "Designing an early warning system for debt crises," Emerging Markets Review, Elsevier, vol. 6(4), pages 376-395, December.
    8. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    9. Barrell, Ray & Davis, E. Philip & Karim, Dilruba & Liadze, Iana, 2010. "Bank regulation, property prices and early warning systems for banking crises in OECD countries," Journal of Banking & Finance, Elsevier, vol. 34(9), pages 2255-2264, September.
    10. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    11. Fioramanti, Marco, 2008. "Predicting sovereign debt crises using artificial neural networks: A comparative approach," Journal of Financial Stability, Elsevier, vol. 4(2), pages 149-164, June.
    12. Manasse, Paolo & Roubini, Nouriel, 2009. ""Rules of thumb" for sovereign debt crises," Journal of International Economics, Elsevier, vol. 78(2), pages 192-205, July.
    13. Tobias Knedlik & Gregor Von Schweinitz, 2012. "Macroeconomic Imbalances as Indicators for Debt Crises in Europe," Journal of Common Market Studies, Wiley Blackwell, vol. 50(5), pages 726-745, September.
    14. Pablo Hernández de Cos & Enrique Moral-Benito & Gerrit B. Koester & Christiane Nickel, 2014. "Signalling fiscal stress in the euro area: A country-specific early warning system," Working Papers 1418, Banco de España.
    15. E. Philip Davis & Dilruba Karim, 2008. "Could Early Warning Systems Have Helped To Predict the Sub-Prime Crisis?," National Institute Economic Review, National Institute of Economic and Social Research, vol. 206(1), pages 35-47, October.
    16. Mr. Paul Cashin & Rupa Duttagupta, 2008. "The Anatomy of Banking Crises," IMF Working Papers 2008/093, International Monetary Fund.
    17. Demyanyk, Yuliya & Hasan, Iftekhar, 2010. "Financial crises and bank failures: A review of prediction methods," Omega, Elsevier, vol. 38(5), pages 315-324, October.
    18. Marco Lo Duca & Tuomas Peltonen, 2011. "Macrofinancial vulnerabilities and future financial stress: assessing systemic risks and predicting systemic events," BIS Papers chapters, in: Bank for International Settlements (ed.), Macroprudential regulation and policy, volume 60, pages 82-88, Bank for International Settlements.
    19. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    20. Asli Demirgüç-Kunt & Enrica Detragiache, 2005. "Cross-Country Empirical Studies of Systemic Bank Distress: A Survey," National Institute Economic Review, National Institute of Economic and Social Research, vol. 192(1), pages 68-83, April.
    21. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    22. Carmen M. Reinhart & Kenneth S. Rogoff, 2011. "From Financial Crash to Debt Crisis," American Economic Review, American Economic Association, vol. 101(5), pages 1676-1706, August.
    23. Mr. Axel Schimmelpfennig & Nouriel Roubini & Paolo Manasse, 2003. "Predicting Sovereign Debt Crises," IMF Working Papers 2003/221, International Monetary Fund.
    24. Katia Berti & Matteo Salto & Matthieu Lequien, 2012. "An early-detection index of fiscal stress for EU countries," European Economy - Economic Papers 2008 - 2015 475, Directorate General Economic and Financial Affairs (DG ECFIN), European Commission.
    25. Bluwstein, Kristina & Buckmann, Marcus & Joseph, Andreas & Kang, Miao & Kapadia, Sujit & Simsek, Özgür, 2020. "Credit growth, the yield curve and financial crisis prediction: evidence from a machine learning approach," Bank of England working papers 848, Bank of England.
    26. Fuertes, Ana-Maria & Kalotychou, Elena, 2006. "Early warning systems for sovereign debt crises: The role of heterogeneity," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 1420-1441, November.
    27. Fuertes, Ana-Maria & Kalotychou, Elena, 2007. "Optimal design of early warning systems for sovereign debt crises," International Journal of Forecasting, Elsevier, vol. 23(1), pages 85-100.
    28. Juan J. Cruces & Christoph Trebesch, 2013. "Sovereign Defaults: The Price of Haircuts," American Economic Journal: Macroeconomics, American Economic Association, vol. 5(3), pages 85-117, July.
    29. David Beers & Jean-Sébastien Nadeau, 2014. "Database of Sovereign Defaults, 2015 (Revised May 2015)," Technical Reports 101, Bank of Canada.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    2. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.
    3. Rho, Caterina & Saenz, Manrique, 2021. "Financial stress and the probability of sovereign default," Journal of International Money and Finance, Elsevier, vol. 110(C).
    4. Filippopoulou, Chryssanthi & Galariotis, Emilios & Spyrou, Spyros, 2020. "An early warning system for predicting systemic banking crises in the Eurozone: A logit regression approach," Journal of Economic Behavior & Organization, Elsevier, vol. 172(C), pages 344-363.
    5. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    6. Mr. Jiro Honda & Rene Tapsoba & Ismael Issifou, 2018. "When Do We Repair the Roof? Insights from Responses to Fiscal Crisis Early Warning Signals," IMF Working Papers 2018/077, International Monetary Fund.
    7. Martin Bruns & Tigran Poghosyan, 2018. "Leading indicators of fiscal distress: evidence from extreme bounds analysis," Applied Economics, Taylor & Francis Journals, vol. 50(13), pages 1454-1478, March.
    8. Medas, Paulo & Poghosyan, Tigran & Xu, Yizhi & Farah-Yacoub, Juan & Gerling, Kerstin, 2018. "Fiscal crises," Journal of International Money and Finance, Elsevier, vol. 88(C), pages 191-207.
      • Mr. Paulo A Medas & Mr. Tigran Poghosyan & Mrs. Kerstin Gerling & Yizhi Xu & Juan Farah-Yacoub, 2017. "Fiscal Crises," IMF Working Papers 2017/086, International Monetary Fund.
    9. Betz, Frank & Oprică, Silviu & Peltonen, Tuomas A. & Sarlin, Peter, 2014. "Predicting distress in European banks," Journal of Banking & Finance, Elsevier, vol. 45(C), pages 225-241.
    10. Roberto Savona & Marika Vezzoli, 2012. "Multidimensional Distance‐To‐Collapse Point And Sovereign Default Prediction," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 19(4), pages 205-228, October.
    11. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    12. Geraldine Dany-Knedlik & Martina Kämpfe & Tobias Knedlik, 2021. "The appropriateness of the macroeconomic imbalance procedure for Central and Eastern European Countries," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 48(1), pages 123-139, February.
    13. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    14. Mathonnat, Clément & Minea, Alexandru, 2018. "Financial development and the occurrence of banking crises," Journal of Banking & Finance, Elsevier, vol. 96(C), pages 344-354.
    15. Sarlin, Peter & Peltonen, Tuomas A., 2013. "Mapping the state of financial stability," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 26(C), pages 46-76.
    16. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    17. Fu, Junhui & Zhou, Qingling & Liu, Yufang & Wu, Xiang, 2020. "Predicting stock market crises using daily stock market valuation and investor sentiment indicators," The North American Journal of Economics and Finance, Elsevier, vol. 51(C).
    18. Fendel Ralf & Stremmel Hanno, 2016. "Characteristics of Banking Crises: A Comparative Study with Geographical Contagion," Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), De Gruyter, vol. 236(3), pages 349-388, May.
    19. Stijn Claessens & M. Ayhan Kose, 2013. "Financial Crises: Explanations, Types and Implications," CAMA Working Papers 2013-06, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    20. Sarlin, Peter & von Schweinitz, Gregor, 2021. "Optimizing Policymakers’ Loss Functions In Crisis Prediction: Before, Within Or After?," Macroeconomic Dynamics, Cambridge University Press, vol. 25(1), pages 100-123, January.

    More about this item

    Keywords

    early warning system; interpretability of machine learning; predictive performance;
    All these keywords.

    JEL classification:

    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • C53 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Forecasting and Prediction Models; Simulation Methods
    • H63 - Public Economics - - National Budget, Deficit, and Debt - - - Debt; Debt Management; Sovereign Debt
    • G01 - Financial Economics - - General - - - Financial Crises

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ecb:ecbwps:20202408. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/emieude.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Official Publications (email available below). General contact details of provider: https://edirc.repec.org/data/emieude.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.