IDEAS home Printed from https://ideas.repec.org/p/zbw/esprep/251726.html
   My bibliography  Save this paper

Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data

Author

Listed:
  • Yildirim, Yusuf
  • Sanyal, Anirban

Abstract

Early warning indicators (EWIs) of banking crises should ideally be judged on how well they function in relation to the choice issue faced by macroprudential policymakers. Several practical features of this challenge are translated into statistical evaluation criteria, including difficulties measuring the costs and advantages of various policy interventions, as well as requirements for the timeliness and stability of EWIs. We analyze the balance panel of possible EWIs for six countries that have experienced currency crisis and banking crisis in recent times. Using possible early warning indicators, we evaluate the suitability of these EWIs in view of their predictive power and stability of signals. The paper observes that credit disbursements to non-financial sectors and central government provides stable signal about systemic risks. Further debt service ratio, inter bank rates and total reserves are also found to be useful in predicting these crisis. Lastly, the paper observes that linear combination of these indicators improves the predictive power of EWIs further.

Suggested Citation

  • Yildirim, Yusuf & Sanyal, Anirban, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," EconStor Preprints 251726, ZBW - Leibniz Information Centre for Economics.
    • Handle: RePEc:zbw:esprep:251726
    DOI: 10.2139/ssrn.4041666
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/251726/1/roccurve8.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.2139/ssrn.4041666?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Andrew Berg & Eduardo Borensztein & Catherine Pattillo, 2005. "Assessing Early Warning Systems: How Have They Worked in Practice?," IMF Staff Papers, Palgrave Macmillan, vol. 52(3), pages 1-5.
    3. Babecký, Jan & Havránek, Tomáš & Matějů, Jakub & Rusnák, Marek & Šmídková, Kateřina & Vašíček, Bořek, 2014. "Banking, debt, and currency crises in developed countries: Stylized facts and early warning indicators," Journal of Financial Stability, Elsevier, vol. 15(C), pages 1-17.
    4. Jahn, Nadya & Kick, Thomas, 2012. "Early warning indicators for the German banking system: A macroprudential analysis," Discussion Papers 27/2012, Deutsche Bundesbank.
    5. Reinhart, Carmen & Rogoff, Kenneth, 2009. "This Time It’s Different: Eight Centuries of Financial Folly-Preface," MPRA Paper 17451, University Library of Munich, Germany.
    6. Carmen M. Reinhart & Graciela L. Kaminsky, 1999. "The Twin Crises: The Causes of Banking and Balance-of-Payments Problems," American Economic Review, American Economic Association, vol. 89(3), pages 473-500, June.
    7. Carmen M. Reinhart & Kenneth S. Rogoff, 2009. "Varieties of Crises and Their Dates," Introductory Chapters, in: This Time Is Different: Eight Centuries of Financial Folly, Princeton University Press.
    8. Baker, Stuart G. & Kramer, Barnett S., 2007. "Peirce, Youden, and Receiver Operating Characteristic Curves," The American Statistician, American Statistical Association, vol. 61, pages 343-346, November.
    9. Christian Dembiermont & Mathias Drehmann & Siriporn Muksakunratana, 2013. "How much does the private sector really borrow - a new database for total credit to the private non-financial sector," BIS Quarterly Review, Bank for International Settlements, March.
    10. Reinhart, Carmen & Rogoff, Kenneth, 2009. "This Time It’s Different: Eight Centuries of Financial Folly-Chapter 1," MPRA Paper 17452, University Library of Munich, Germany.
    11. Özgür Orhangazi, 2014. "Capital Flows and Credit Expansions in Turkey," Review of Radical Political Economics, Union for Radical Political Economics, vol. 46(4), pages 509-516, December.
    12. Alessi, Lucia & Detken, Carsten, 2011. "Quasi real time early warning indicators for costly asset price boom/bust cycles: A role for global liquidity," European Journal of Political Economy, Elsevier, vol. 27(3), pages 520-533, September.
    13. Òscar Jordà & Moritz Schularick & Alan M Taylor, 2011. "Financial Crises, Credit Booms, and External Imbalances: 140 Years of Lessons," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 59(2), pages 340-378, June.
    14. Mark David Witte, 2012. "What Effect Did the Credit Crisis of 2008 Have on European Exchange Rates?," Eastern European Economics, Taylor & Francis Journals, vol. 50(3), pages 79-93, May.
    15. Yu, Wenbao & Park, Taesung, 2015. "Two simple algorithms on linear combination of multiple biomarkers to maximize partial area under the ROC curve," Computational Statistics & Data Analysis, Elsevier, vol. 88(C), pages 15-27.
    16. Jan Babecký & Tomáš Havránek & Jakub Matìjù & Marek Rusnák & Kateøina Šmídková & Boøek Vašíèek, 2011. "Early Warning Indicators of Crisis Incidence: Evidence from a Panel of 40 Developed Countries," Working Papers IES 2011/36, Charles University Prague, Faculty of Social Sciences, Institute of Economic Studies, revised Nov 2011.
    17. Graciela Kaminsky & Saul Lizondo & Carmen M. Reinhart, 1998. "Leading Indicators of Currency Crises," IMF Staff Papers, Palgrave Macmillan, vol. 45(1), pages 1-48, March.
    18. Yusuf Yıldırım & Anirban Sanyal, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 69(4), pages 557-597, December.
    19. Ms. Sally Chen & Katsiaryna Svirydzenka, 2021. "Financial Cycles – Early Warning Indicators of Banking Crises?," IMF Working Papers 2021/116, International Monetary Fund.
    20. Joon-Ho Hahm & Hyun Song Shin & Kwanho Shin, 2013. "Noncore Bank Liabilities and Financial Vulnerability," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 45, pages 3-36, August.
    21. repec:rnp:ecopol:09111 is not listed on IDEAS
    22. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    23. Park, Cyn-Young & Mercado, Rogelio V., 2014. "Determinants of financial stress in emerging market economies," Journal of Banking & Finance, Elsevier, vol. 45(C), pages 199-224.
    24. Mathias Drehmann & Mikael Juselius, 2012. "Do debt service costs affect macroeconomic and financial stability?," BIS Quarterly Review, Bank for International Settlements, September.
    25. Dornbusch, Rudiger, 1976. "Expectations and Exchange Rate Dynamics," Journal of Political Economy, University of Chicago Press, vol. 84(6), pages 1161-1176, December.
    26. Thibaut Duprey & Benjamin Klaus, 2017. "How to Predict Financial Stress? An Assessment of Markov Switching Models," Staff Working Papers 17-32, Bank of Canada.
    27. Margaret S. Pepe & Gary Longton & Holly Janes, 2009. "Estimation and comparison of receiver operating characteristic curves," Stata Journal, StataCorp LLC, vol. 9(1), pages 1-16, March.
    28. Ravi Balakrishnan & Stephan Danninger & Selim Elekdag & Irina Tytell, 2011. "The Transmission of Financial Stress from Advanced to Emerging Economies," Emerging Markets Finance and Trade, Taylor & Francis Journals, vol. 47(0), pages 40-68, May.
    29. Mathias Drehmann & Claudio Borio & Kostas Tsatsaronis, 2011. "Anchoring Countercyclical Capital Buffers: The role of Credit Aggregates," International Journal of Central Banking, International Journal of Central Banking, vol. 7(4), pages 189-240, December.
    30. Pierre-Olivier Gourinchas & Maurice Obstfeld, 2012. "Stories of the Twentieth Century for the Twenty-First," American Economic Journal: Macroeconomics, American Economic Association, vol. 4(1), pages 226-265, January.
    31. Marcin Pietrzak, 2021. "Can Financial Soundness Indicators Help Predict Financial Sector Distress?," IMF Working Papers 2021/197, International Monetary Fund.
    32. Ilja Viktorov & Alexander Abramov, 2020. "The 2014–15 Financial Crisis in Russia and the Foundations of Weak Monetary Power Autonomy in the International Political Economy," New Political Economy, Taylor & Francis Journals, vol. 25(4), pages 487-510, June.
    33. Alvaro Ortiz Vidal-Abarca & Alfonso Ugarte Ruiz, 2015. "Introducing a New Early Warning System Indicator (EWSI) of banking crises," Working Papers 1502, BBVA Bank, Economic Research Department.
    34. Santiago Fernandez de Lis & Alicia Garcia-Herrero, 2013. "Dynamic Provisioning: A Buffer Rather Than a Countercyclical Tool?," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 35-67.
    35. Santiago Fernandez de Lis & Alicia Garcia-Herrero, 2013. "Dynamic Provisioning: A Buffer Rather Than a Countercyclical Tool?," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Spring 20), pages 35-67.
    36. Geršl, Adam & Jašová, Martina, 2018. "Credit-based early warning indicators of banking crises in emerging markets," Economic Systems, Elsevier, vol. 42(1), pages 18-31.
    37. Martínez, Juan Francisco & Oda, Daniel, 2021. "Characterization of the Chilean financial cycle, early warning indicators and implications for macro-prudential policies," Latin American Journal of Central Banking (previously Monetaria), Elsevier, vol. 2(1).
    38. Simone Romano, 2021. "The 2011 Crisis in Italy: A Story of Deep-Rooted (and Still Unresolved) Economic and Political Weaknesses," Springer Books, in: Bettina De Souza Guilherme & Christian Ghymers & Stephany Griffith-Jones & Andrea Ribeiro Hoffmann (ed.), Financial Crisis Management and Democracy, edition 1, chapter 0, pages 173-184, Springer.
    39. Cohen, Jacqueline & Garman, Samuel & Gorr, Wilpen, 2009. "Empirical calibration of time series monitoring methods using receiver operating characteristic curves," International Journal of Forecasting, Elsevier, vol. 25(3), pages 484-497, July.
    40. Travis J. Berge & Òscar Jordà, 2011. "Evaluating the Classification of Economic Activity into Recessions and Expansions," American Economic Journal: Macroeconomics, American Economic Association, vol. 3(2), pages 246-277, April.
    41. Orsolya Csortos & Zoltán Szalai, 2014. "Early warning indicators: financial and macroeconomic imbalances in Central and Eastern European countries," MNB Working Papers 2014/2, Magyar Nemzeti Bank (Central Bank of Hungary).
    42. Mathias Drehmann & Claudio Borio & Leonardo Gambacorta & Gabriel Jiminez & Carlos Trucharte, 2010. "Countercyclical capital buffers: exploring options," BIS Working Papers 317, Bank for International Settlements.
    43. Berg, Andrew & Pattillo, Catherine, 1999. "Predicting currency crises:: The indicators approach and an alternative," Journal of International Money and Finance, Elsevier, vol. 18(4), pages 561-586, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yusuf Yıldırım & Anirban Sanyal, 2022. "Evaluating the Effectiveness of Early Warning Indicators: An Application of Receiver Operating Characteristic Curve Approach to Panel Data," Scientific Annals of Economics and Business (continues Analele Stiintifice), Alexandru Ioan Cuza University, Faculty of Economics and Business Administration, vol. 69(4), pages 557-597, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Drehmann, Mathias & Juselius, Mikael, 2014. "Evaluating early warning indicators of banking crises: Satisfying policy requirements," International Journal of Forecasting, Elsevier, vol. 30(3), pages 759-780.
    2. Alessi, Lucia & Detken, Carsten, 2018. "Identifying excessive credit growth and leverage," Journal of Financial Stability, Elsevier, vol. 35(C), pages 215-225.
    3. Ebrahimi Kahou, Mahdi & Lehar, Alfred, 2017. "Macroprudential policy: A review," Journal of Financial Stability, Elsevier, vol. 29(C), pages 92-105.
    4. Tihana Skrinjaric, 2023. "Leading indicators of financial stress in Croatia: a regime switching approach," Public Sector Economics, Institute of Public Finance, vol. 47(2), pages 205-232.
    5. André K. Anundsen & Karsten Gerdrup & Frank Hansen & Kasper Kragh‐Sørensen, 2016. "Bubbles and Crises: The Role of House Prices and Credit," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 31(7), pages 1291-1311, November.
    6. Markus Behn & Carsten Detken & Tuomas Peltonen & Willem Schudel, 2017. "Predicting Vulnerabilities in the EU Banking Sector: The Role of Global and Domestic Factors," International Journal of Central Banking, International Journal of Central Banking, vol. 13(4), pages 147-189, December.
    7. Carsten Detken & Olaf Weeken & Lucia Alessi & Diana Bonfim & Miguel M. Boucinha & Christian Castro & Sebastian Frontczak & Gaston Giordana & Julia Giese & Nadya Jahn & Jan Kakes & Benjamin Klaus & Jan, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 05, European Systemic Risk Board.
    8. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2018. "An evaluation of early warning models for systemic banking crises: Does machine learning improve predictions?," Discussion Papers 48/2018, Deutsche Bundesbank.
    9. Schudel, Willem, 2015. "Shifting horizons: assessing macro trends before, during, and following systemic banking crises," Working Paper Series 1766, European Central Bank.
    10. Jon Danielsson & Marcela Valenzuela & Ilknur Zer, 2018. "Learning from History: Volatility and Financial Crises," The Review of Financial Studies, Society for Financial Studies, vol. 31(7), pages 2774-2805.
    11. Detken, Carsten & Weeken, Olaf & Alessi, Lucia & Bonfim, Diana & Boucinha, Miguel & Castro, Christian & Frontczak, Sebastian & Giordana, Gaston & Giese, Julia & Wildmann, Nadya & Kakes, Jan & Klaus, B, 2014. "Operationalising the countercyclical capital buffer: indicator selection, threshold identification and calibration options," ESRB Occasional Paper Series 5, European Systemic Risk Board.
    12. Detken, Carsten & Peltonen, Tuomas A. & Schudel, Willem & Behn, Markus, 2013. "Setting countercyclical capital buffers based on early warning models: would it work?," Working Paper Series 1604, European Central Bank.
    13. Helene Olsen & Harald Wieslander, 2020. "The Impact of Monetary Policy on Leading Variables for Financial Stability in Norway," Working Papers No 02/2020, Centre for Applied Macro- and Petroleum economics (CAMP), BI Norwegian Business School.
    14. Xianglong Liu, 2023. "Towards Better Banking Crisis Prediction: Could an Automatic Variable Selection Process Improve the Performance?," The Economic Record, The Economic Society of Australia, vol. 99(325), pages 288-312, June.
    15. Kauko, Karlo, 2014. "How to foresee banking crises? A survey of the empirical literature," Economic Systems, Elsevier, vol. 38(3), pages 289-308.
    16. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.
    17. Antunes, António & Bonfim, Diana & Monteiro, Nuno & Rodrigues, Paulo M.M., 2018. "Forecasting banking crises with dynamic panel probit models," International Journal of Forecasting, Elsevier, vol. 34(2), pages 249-275.
    18. Beutel, Johannes & List, Sophia & von Schweinitz, Gregor, 2019. "Does machine learning help us predict banking crises?," Journal of Financial Stability, Elsevier, vol. 45(C).
    19. Gertler, Pavel & Hofmann, Boris, 2018. "Monetary facts revisited," Journal of International Money and Finance, Elsevier, vol. 86(C), pages 154-170.
    20. Geršl, Adam & Jašová, Martina, 2018. "Credit-based early warning indicators of banking crises in emerging markets," Economic Systems, Elsevier, vol. 42(1), pages 18-31.

    More about this item

    Keywords

    EWIs; ROC; area under the curve; shrinkage;
    All these keywords.

    JEL classification:

    • G01 - Financial Economics - - General - - - Financial Crises
    • C40 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods: Special Topics - - - General
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:esprep:251726. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://edirc.repec.org/data/zbwkide.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.