IDEAS home Printed from https://ideas.repec.org/p/zbw/esprep/251726.html
   My bibliography  Save this paper

Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data

Author

Listed:
  • Yildirim, Yusuf
  • Sanyal, Anirban

Abstract

Early warning indicators (EWIs) of banking crises should ideally be judged on how well they function in relation to the choice issue faced by macroprudential policymakers. Several practical features of this challenge are translated into statistical evaluation criteria, including difficulties measuring the costs and advantages of various policy interventions, as well as requirements for the timeliness and stability of EWIs. We analyze the balance panel of possible EWIs for six countries that have experienced currency crisis and banking crisis in recent times. Using possible early warning indicators, we evaluate the suitability of these EWIs in view of their predictive power and stability of signals. The paper observes that credit disbursements to non-financial sectors and central government provides stable signal about systemic risks. Further debt service ratio, inter bank rates and total reserves are also found to be useful in predicting these crisis. Lastly, the paper observes that linear combination of these indicators improves the predictive power of EWIs further.

Suggested Citation

  • Yildirim, Yusuf & Sanyal, Anirban, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," EconStor Preprints 251726, ZBW - Leibniz Information Centre for Economics.
    • Handle: RePEc:zbw:esprep:251726
    DOI: 10.2139/ssrn.4041666
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/251726/1/roccurve8.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.2139/ssrn.4041666?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Carmen M. Reinhart & Kenneth S. Rogoff, 2014. "This Time is Different: A Panoramic View of Eight Centuries of Financial Crises," Annals of Economics and Finance, Society for AEF, vol. 15(2), pages 215-268, November.
    3. Andrew Berg & Eduardo Borensztein & Catherine Pattillo, 2005. "Assessing Early Warning Systems: How Have They Worked in Practice?," IMF Staff Papers, Palgrave Macmillan, vol. 52(3), pages 1-5.
    4. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    5. Jahn, Nadya & Kick, Thomas, 2012. "Early warning indicators for the German banking system: A macroprudential analysis," Discussion Papers 27/2012, Deutsche Bundesbank.
    6. Reinhart, Carmen & Rogoff, Kenneth, 2009. "This Time It’s Different: Eight Centuries of Financial Folly-Preface," MPRA Paper 17451, University Library of Munich, Germany.
    7. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    8. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Varieties of Crises and Their Dates," Introductory Chapters, in: This Time Is Different: Eight Centuries of Financial Folly, Princeton University Press.
    9. Baker, Stuart G. & Kramer, Barnett S., 2007. "Peirce, Youden, and Receiver Operating Characteristic Curves," The American Statistician, American Statistical Association, vol. 61, pages 343-346, November.
    10. Christian Dembiermont & Mathias Drehmann & Siriporn Muksakunratana, 2013. "How much does the private sector really borrow - a new database for total credit to the private non-financial sector," BIS Quarterly Review, Bank for International Settlements, March.
    11. Reinhart, Carmen & Rogoff, Kenneth, 2009. "This Time It’s Different: Eight Centuries of Financial Folly-Chapter 1," MPRA Paper 17452, University Library of Munich, Germany.
    12. Özgür Orhangazi, 2014. "Capital Flows and Credit Expansions in Turkey," Review of Radical Political Economics, Union for Radical Political Economics, vol. 46(4), pages 509-516, December.
    13. Òscar Jordà & Moritz Schularick & Alan M Taylor, 2011. "Financial Crises, Credit Booms, and External Imbalances: 140 Years of Lessons," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 59(2), pages 340-378, June.
    14. Mark David Witte, 2012. "What Effect Did the Credit Crisis of 2008 Have on European Exchange Rates?," Eastern European Economics, Taylor & Francis Journals, vol. 50(3), pages 79-93, May.
    15. Yu, Wenbao & Park, Taesung, 2015. "Two simple algorithms on linear combination of multiple biomarkers to maximize partial area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 88(C), pages 15-27.
    16. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    17. Yusuf Yıldırım & Anirban Sanyal, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 69(4), pages 557-597, December.
    18. Ms. Sally Chen & Katsiaryna Svirydzenka, 2021. "Financial Cycles – Early Warning Indicators of Banking Crises?," IMF Working Papers 2021/116, International Monetary Fund.
    19. Joon-Ho Hahm & Hyun Song Shin & Kwanho Shin, 2013. "Noncore Bank Liabilities and Financial Vulnerability," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45, pages 3-36, August.
    20. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    21. Mathias Drehmann & Mikael Juselius, 2012. "Do debt service costs affect macroeconomic and financial stability?," BIS Quarterly Review, Bank for International Settlements, September.
    22. Dornbusch, Rudiger, 1976. "Expectations and Exchange Rate Dynamics," Journal of Political Economy, University of Chicago Press, vol. 84(6), pages 1161-1176, December.
    23. Pierre-Olivier Gourinchas & Maurice Obstfeld, 2012. "Stories of the Twentieth Century for the Twenty-First," American Economic Journal: Macroeconomics, American Economic Association, vol. 4(1), pages 226-265, January.
    24. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2011. "Anchoring Countercyclical Capital Buffers: The role of Credit Aggregates," International Journal of Central Banking, International Journal of Central Banking, vol. 7(4), pages 189-240, December.
    25. Ilja Viktorov & Alexander Abramov, 2020. "The 2014–15 Financial Crisis in Russia and the Foundations of Weak Monetary Power Autonomy in the International Political Economy," New Political Economy, Taylor & Francis Journals, vol. 25(4), pages 487-510, June.
    26. Geršl, Adam & Jašová, Martina, 2018. "Credit-based early warning indicators of banking crises in emerging markets," Economic Systems, Elsevier, vol. 42(1), pages 18-31.
    27. Martínez, Juan Francisco & Oda, Daniel, 2021. "Characterization of the Chilean financial cycle, early warning indicators and implications for macro-prudential policies," Latin American Journal of Central Banking (previously Monetaria), Elsevier, vol. 2(1).
    28. Simone Romano, 2021. "The 2011 Crisis in Italy: A Story of Deep-Rooted (and Still Unresolved) Economic and Political Weaknesses," Springer Books, in: Bettina De Souza Guilherme & Christian Ghymers & Stephany Griffith-Jones & Andrea Ribeiro Hoffmann (ed.), Financial Crisis Management and Democracy, edition 1, chapter 0, pages 173-184, Springer.
    29. Cohen, Jacqueline & Garman, Samuel & Gorr, Wilpen, 2009. "Empirical calibration of time series monitoring methods using receiver operating characteristic curves," International Journal of Forecasting, Elsevier, vol. 25(3), pages 484-497, July.
    30. Travis J. Berge & Òscar Jordà, 2011. "Evaluating the Classification of Economic Activity into Recessions and Expansions," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(2), pages 246-277, April.
    31. Mathias Drehmann & Claudio Borio & Leonardo Gambacorta & Gabriel Jiminez & Carlos Trucharte, 2010. "Countercyclical capital buffers: exploring options," BIS Working Papers 317, Bank for International Settlements.
    32. Berg, Andrew & Pattillo, Catherine, 1999. "Predicting currency crises:: The indicators approach and an alternative," Journal of International Money and Finance, Elsevier, vol. 18(4), pages 561-586, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yusuf Yıldırım & Anirban Sanyal, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 69(4), pages 557-597, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    3. Tihana Skrinjaric, 2023. "Leading indicators of financial stress in Croatia: a regime switching approach," Public Sector Economics, Institute of Public Finance, vol. 47(2), pages 205-232.
    4. Jon Danielsson & Marcela Valenzuela & Ilknur Zer, 2018. "Learning from History: Volatility and Financial Crises," The Review of Financial Studies, Society for Financial Studies, vol. 31(7), pages 2774-2805.
    5. Ebrahimi Kahou, Mahdi & Lehar, Alfred, 2017. "Macroprudential policy: A review," Journal of Financial Stability, Elsevier, vol. 29(C), pages 92-105.
    6. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2018. "An evaluation of early warning models for systemic banking crises: Does machine learning improve predictions?," Discussion Papers 48/2018, Deutsche Bundesbank.
    7. Gertler, Pavel & Hofmann, Boris, 2018. "Monetary facts revisited," Journal of International Money and Finance, Elsevier, vol. 86(C), pages 154-170.
    8. André K. Anundsen & Karsten Gerdrup & Frank Hansen & Kasper Kragh‐Sørensen, 2016. "Bubbles and Crises: The Role of House Prices and Credit," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1291-1311, November.
    9. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    10. Bordo, M.D. & Meissner, C.M., 2016. "Fiscal and Financial Crises," Handbook of Macroeconomics, in: J. B. Taylor & Harald Uhlig (ed.), Handbook of Macroeconomics, edition 1, volume 2, chapter 0, pages 355-412, Elsevier.
    11. Xianglong Liu, 2023. "Towards Better Banking Crisis Prediction: Could an Automatic Variable Selection Process Improve the Performance?," The Economic Record, The Economic Society of Australia, vol. 99(325), pages 288-312, June.
    12. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    13. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
    14. Catão, Luis A.V. & Milesi-Ferretti, Gian Maria, 2014. "External liabilities and crises," Journal of International Economics, Elsevier, vol. 94(1), pages 18-32.
    15. Schudel, Willem, 2015. "Shifting horizons: assessing macro trends before, during, and following systemic banking crises," Working Paper Series 1766, European Central Bank.
    16. Kim Ristolainen, 2018. "Predicting Banking Crises with Artificial Neural Networks: The Role of Nonlinearity and Heterogeneity," Scandinavian Journal of Economics, Wiley Blackwell, vol. 120(1), pages 31-62, January.
    17. Markus Holopainen & Peter Sarlin, 2015. "Toward robust early-warning models: A horse race, ensembles and model uncertainty," Papers 1501.04682, arXiv.org, revised Apr 2016.
    18. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    19. Chung‐Hua Shen & Hsing‐Hua Hsu, 2022. "The determinants of Asian banking crises—Application of the panel threshold logit model," International Review of Finance, International Review of Finance Ltd., vol. 22(1), pages 248-277, March.
    20. Antunes, António & Bonfim, Diana & Monteiro, Nuno & Rodrigues, Paulo M.M., 2018. "Forecasting banking crises with dynamic panel probit models," International Journal of Forecasting, Elsevier, vol. 34(2), pages 249-275.

    More about this item

    Keywords

    EWIs; ROC; area under the curve; shrinkage;
    All these keywords.

    JEL classification:

    • G01 - Financial Economics - - General - - - Financial Crises
    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:esprep:251726. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.