IDEAS home Printed from https://ideas.repec.org/a/nbp/nbpbik/v47y2016i3p227-250.html
   My bibliography  Save this article

Rank-order statistics for validating discriminative power of credit risk models

Author

Listed:
  • Lukasz Prorokowski

    (H.L. Prorokowski LLC, San Francisco)

Abstract

This paper provides practical insights into common statistical measures used to validate a model’s discriminatory power for the probability of default (PD), loss liven default (LGD) and exposure at default (EAD). The review of available rank-order statistics is not based on analysing empirical data. Thus, the study has more of an informative value without delivering empirical evidence. When there is an alternative model available for comparison, this paper proposes to use the cumulative accuracy curve and the accuracy ratio to assess the rank-order ability for PD models given their popularity in practice. When there is no model available for comparison, due to the limited techniques in this area, this paper proposes to compare the confidence intervals in order to prove that a rating system has any discriminative power. For the LGD/EAD/slotting models, this paper recommends using a graph to check the rank-order ability. No statistical test is recommended. Focusing on enhancing practical implications for the financial industry, this paper advises banks on the existing CRR self-attestation requirements.

Suggested Citation

  • Lukasz Prorokowski, 2016. "Rank-order statistics for validating discriminative power of credit risk models," Bank i Kredyt, Narodowy Bank Polski, vol. 47(3), pages 227-250.
    • Handle: RePEc:nbp:nbpbik:v:47:y:2016:i:3:p:227-250
    as

    Download full text from publisher

    File URL: https://bankikredyt.nbp.pl/content/2016/03/bik_03_2016_02_art.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Stephen Satchel & Wei Xia, 2006. "Analytic Models of the ROC Curve: Applications to Credit Rating Model Validation," Research Paper Series 181, Quantitative Finance Research Centre, University of Technology, Sydney.
    2. Engelmann, Bernd & Hayden, Evelyn & Tasche, Dirk, 2003. "Measuring the Discriminative Power of Rating Systems," Discussion Paper Series 2: Banking and Financial Studies 2003,01, Deutsche Bundesbank.
    3. Dirk Tasche, 2009. "Estimating discriminatory power and PD curves when the number of defaults is small," Papers 0905.3928, arXiv.org, revised Mar 2010.
    4. Newson, Roger, 2005. "Efficient Calculation of Jackknife Confidence Intervals for Rank Statistics," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 15(i01).
    5. Roger Newson, 2002. "Parameters behind "nonparametric" statistics: Kendall's tau,Somers' D and median differences," Stata Journal, StataCorp LLC, vol. 2(1), pages 45-64, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Nehrebecka Natalia, 2018. "An Evaluation of the Discriminatory Power of Selected Polish Bankruptcy Prediction Models As Part of the Validation Process," Financial Sciences. Nauki o Finansach, Sciendo, vol. 23(4), pages 63-88, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dirk Tasche, 2009. "Estimating discriminatory power and PD curves when the number of defaults is small," Papers 0905.3928, arXiv.org, revised Mar 2010.
    2. M. V. Pomazanov, 2022. "Second-order accuracy metrics for scoring models and their practical use," Papers 2204.07989, arXiv.org, revised Nov 2022.
    3. Peter, Eckley, 2015. "Measuring economic uncertainty using news-media textual data," MPRA Paper 64874, University Library of Munich, Germany, revised 01 May 2015.
    4. Wosnitza, Jan Henrik, 2022. "Calibration alternatives to logistic regression and their potential for transferring the dispersion of discriminatory power into uncertainties of probabilities of default," Discussion Papers 04/2022, Deutsche Bundesbank.
    5. Kristóf, Tamás, 2008. "A csődelőrejelzés és a nem fizetési valószínűség számításának módszertani kérdéseiről [Some methodological questions of bankruptcy prediction and probability of default estimation]," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(5), pages 441-461.
    6. John, Katrin & Thomsen, Stephan L., 2015. "School-track environment or endowment: What determines different other-regarding behavior across peer groups?," Games and Economic Behavior, Elsevier, vol. 94(C), pages 122-141.
    7. Dirk Tasche, . "Fitting a distribution to value-at-risk and expected shortfall, with an application to covered bonds," Journal of Credit Risk, Journal of Credit Risk.
    8. Dean Fantazzini & Silvia Figini, 2009. "Random Survival Forests Models for SME Credit Risk Measurement," Methodology and Computing in Applied Probability, Springer, vol. 11(1), pages 29-45, March.
    9. Roger B. Newson, 2010. "Comparing the predictive powers of survival models using Harrell’s C or Somers’ D," Stata Journal, StataCorp LLC, vol. 10(3), pages 339-358, September.
    10. Chakrabarty, Manisha & Hildenbrand, Werner, 2011. "Engel's Law Reconsidered," Journal of Mathematical Economics, Elsevier, vol. 47(3), pages 289-299.
    11. Catão, Luis A.V. & Milesi-Ferretti, Gian Maria, 2014. "External liabilities and crises," Journal of International Economics, Elsevier, vol. 94(1), pages 18-32.
    12. Guerra, Alice & Zhuravleva, Tatyana, 2021. "Do bystanders react to bribery?," Journal of Economic Behavior & Organization, Elsevier, vol. 185(C), pages 442-462.
    13. Moro Russ A. & Härdle Wolfgang K. & Schäfer Dorothea, 2017. "Company rating with support vector machines," Statistics & Risk Modeling, De Gruyter, vol. 34(1-2), pages 55-67, June.
    14. En-Der Su & Shih-Ming Huang, 2010. "Comparing Firm Failure Predictions Between Logit, KMV, and ZPP Models: Evidence from Taiwan’s Electronics Industry," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 17(3), pages 209-239, September.
    15. John Ly & Vidiya Sathananthan & Thomas Griffiths & Zahir Kanjee & Avi Kenny & Nicholas Gordon & Gaurab Basu & Dale Battistoli & Lorenzo Dorr & Breeanna Lorenzen & Dana R Thomson & Ami Waters & Uriah G, 2016. "Facility-Based Delivery during the Ebola Virus Disease Epidemic in Rural Liberia: Analysis from a Cross-Sectional, Population-Based Household Survey," PLOS Medicine, Public Library of Science, vol. 13(8), pages 1-17, August.
    16. Ramasubramanian Sundararajan & Tarun Bhaskar & Abhinanda Sarkar & Sridhar Dasaratha & Debasis Bal & Jayanth K. Marasanapalle & Beata Zmudzka & Karolina Bak, 2011. "Marketing Optimization in Retail Banking," Interfaces, INFORMS, vol. 41(5), pages 485-505, October.
    17. Elisa Ughetto & Andrea Vezzulli, 2011. "What role can mutual guarantee consortia play for financing innovation? A firm-level study for Italy," International Journal of Banking, Accounting and Finance, Inderscience Enterprises Ltd, vol. 3(4), pages 294-319.
    18. Philipp Chapkovski & Luca Corazzini & Valeria Maggian, 2021. "Does Whistleblowing on Tax Evaders Reduce Ingroup Cooperation?," Working Papers 2021:20, Department of Economics, University of Venice "Ca' Foscari".
    19. Tasche, Dirk, 2013. "Bayesian estimation of probabilities of default for low default portfolios," Journal of Risk Management in Financial Institutions, Henry Stewart Publications, vol. 6(3), pages 302-326, July.
    20. Cingl, Lubomír & Lichard, Tomáš & Miklánek, Tomáš, 2023. "Tax designation effects on compliance: An online experiment with taxpayers," Journal of Economic Behavior & Organization, Elsevier, vol. 214(C), pages 615-633.

    More about this item

    Keywords

    credit risk; rank-order statistics; PD/LGD/EAD validation; CRR (CRD IV); AIRB banks;
    All these keywords.

    JEL classification:

    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • G10 - Financial Economics - - General Financial Markets - - - General (includes Measurement and Data)
    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages
    • G28 - Financial Economics - - Financial Institutions and Services - - - Government Policy and Regulation
    • G32 - Financial Economics - - Corporate Finance and Governance - - - Financing Policy; Financial Risk and Risk Management; Capital and Ownership Structure; Value of Firms; Goodwill

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbp:nbpbik:v:47:y:2016:i:3:p:227-250. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wojciech Burjanek (email available below). General contact details of provider: https://edirc.repec.org/data/nbpgvpl.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.