IDEAS home Printed from https://ideas.repec.org/p/zbw/bubdp2/2225.html
   My bibliography  Save this paper

Measuring the Discriminative Power of Rating Systems

Author

Listed:
  • Engelmann, Bernd
  • Hayden, Evelyn
  • Tasche, Dirk

Abstract

Assessing the discriminative power of rating systems is an important question to banks and to regulators. In this article we analyze the Cumulative Accuracy Profile (CAP) and the Receiver Operating Characteristic (ROC) which are both commonly used in practice. We give a test-theoretic interpretation for the concavity of the CAP and the ROC curve and demonstrate how this observation can be used for more efficiently exploiting the informational contents of accounting ratios. Furthermore, we show that two popular summary statistics of these concepts, namely the Accuracy Ratio and the area under the ROC curve, contain the same information and we analyse the statistical properties of these measures. We show in detail how to identify accounting ratios with high discriminative power, how to calculate confidence intervals for the area below the ROC curve, and how to test if two rating models validated on the same data set are different. All concepts are illustrated by applications to real data.

Suggested Citation

  • Engelmann, Bernd & Hayden, Evelyn & Tasche, Dirk, 2003. "Measuring the Discriminative Power of Rating Systems," Discussion Paper Series 2: Banking and Financial Studies 2003,01, Deutsche Bundesbank.
  • Handle: RePEc:zbw:bubdp2:2225
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/19726/1/200301dkp_b.pdf
    Download Restriction: no

    References listed on IDEAS

    as
    1. Dirk Tasche, 2002. "Remarks on the monotonicity of default probabilities," Papers cond-mat/0207555, arXiv.org.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rafael Repullo & Jesús Saurina & Carlos Trucharte, 2010. "Mitigating the pro-cyclicality of Basel II," Economic Policy, CEPR;CES;MSH, vol. 25, pages 659-702, October.
    2. Zvika Afik & Ohad Arad & Koresh Galil, 2012. "Using Merton model: an empirical assessment of alternatives," Working Papers 1202, Ben-Gurion University of the Negev, Department of Economics.
    3. Edward Altman & Gabriele Sabato, 2005. "Effects of the New Basel Capital Accord on Bank Capital Requirements for SMEs," Journal of Financial Services Research, Springer;Western Finance Association, vol. 28(1), pages 15-42, October.
    4. En-Der Su & Shih-Ming Huang, 2010. "Comparing Firm Failure Predictions Between Logit, KMV, and ZPP Models: Evidence from Taiwan’s Electronics Industry," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 17(3), pages 209-239, September.
    5. Dierkes, Maik & Erner, Carsten & Langer, Thomas & Norden, Lars, 2013. "Business credit information sharing and default risk of private firms," Journal of Banking & Finance, Elsevier, vol. 37(8), pages 2867-2878.
    6. Maik Dierkes & Carsten Erner & Thomas Langer & Lars Norden, 2012. "Business credit information sharing and default risk of private firms," Mo.Fi.R. Working Papers 64, Money and Finance Research group (Mo.Fi.R.) - Univ. Politecnica Marche - Dept. Economic and Social Sciences.
    7. Laura Auria & Rouslan A. Moro, 2008. "Support Vector Machines (SVM) as a Technique for Solvency Analysis," Discussion Papers of DIW Berlin 811, DIW Berlin, German Institute for Economic Research.
    8. Costeiu, Adrian & Neagu, Florian, 2013. "Bridging the banking sector with the real economy: a financial stability perspective," Working Paper Series 1592, European Central Bank.
    9. Ramasubramanian Sundararajan & Tarun Bhaskar & Abhinanda Sarkar & Sridhar Dasaratha & Debasis Bal & Jayanth K. Marasanapalle & Beata Zmudzka & Karolina Bak, 2011. "Marketing Optimization in Retail Banking," Interfaces, INFORMS, vol. 41(5), pages 485-505, October.
    10. Rodriguez, Adolfo & Trucharte, Carlos, 2007. "Loss coverage and stress testing mortgage portfolios: A non-parametric approach," Journal of Financial Stability, Elsevier, vol. 3(4), pages 342-367, December.
    11. Martin Rezac & Frantisek Rezac, 2011. "How to Measure the Quality of Credit Scoring Models," Czech Journal of Economics and Finance (Finance a uver), Charles University Prague, Faculty of Social Sciences, vol. 61(5), pages 486-507, November.
    12. Alexandros Benos & George Papanastasopoulos, 2005. "Extending the Merton Model: A Hybrid Approach to Assessing Credit Quality," Finance 0505020, University Library of Munich, Germany, revised 18 Nov 2005.
    13. Wolfgang Karl Härdle & Dedy Dwi Prastyo & Christian Hafner, 2012. "Support Vector Machines with Evolutionary Feature Selection for Default Prediction," SFB 649 Discussion Papers SFB649DP2012-030, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    14. Radu Muntean, 2009. "Early Warning Models for Banking Supervision in Romania," Advances in Economic and Financial Research - DOFIN Working Paper Series 39, Bucharest University of Economics, Center for Advanced Research in Finance and Banking - CARFIB.
    15. Xu, Xin, 2013. "Forecasting Bankruptcy with Incomplete Information," MPRA Paper 55024, University Library of Munich, Germany, revised 31 Mar 2014.
    16. Ana Paula Matias Gama & Helena Susana Amaral Geraldes, 2012. "Credit risk assessment and the impact of the New Basel Capital Accord on small and medium-sized enterprises: An empirical analysis," Management Research Review, Emerald Group Publishing, vol. 35(8), pages 727-749, July.
    17. João Fernandes, 2005. "Corporate Credit Risk Modeling: Quantitative Rating System And Probability Of Default Estimation," Finance 0505013, University Library of Munich, Germany.
    18. Han-Hsing Lee & Kuanyu Shih & Kehluh Wang, 2016. "Measuring sovereign credit risk using a structural model approach," Review of Quantitative Finance and Accounting, Springer, vol. 47(4), pages 1097-1128, November.
    19. Stefan Hlawatsch, 2009. "A Framework for LGD Validation of Retail Portfolios," FEMM Working Papers 09025, Otto-von-Guericke University Magdeburg, Faculty of Economics and Management.
    20. Afik, Zvika & Arad, Ohad & Galil, Koresh, 2016. "Using Merton model for default prediction: An empirical assessment of selected alternatives," Journal of Empirical Finance, Elsevier, vol. 35(C), pages 43-67.

    More about this item

    Keywords

    Validation; Rating Models; Credit Analysis;

    JEL classification:

    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:bubdp2:2225. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (ZBW - German National Library of Economics). General contact details of provider: http://edirc.repec.org/data/dbbgvde.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.