IDEAS home Printed from https://ideas.repec.org/p/imf/imfwpa/2012-122.html
   My bibliography  Save this paper

Appraising Credit Ratings: Does the CAP Fit Better than the ROC?

Author

Listed:
  • Mr. R. John Irwin
  • Mr. Timothy C Irwin

Abstract

ROC and CAP analysis are alternative methods for evaluating a wide range of diagnostic systems, including assessments of credit risk. ROC analysis is widely used in many fields, but in finance CAP analysis is more common. We compare the two methods, using as an illustration the ability of the OECD’s country risk ratings to predict whether a country will have a program with the IMF (an indicator of financial distress). ROC and CAP analyses both have the advantage of generating measures of accuracy that are independent of the choice of diagnostic threshold, such as risk rating. ROC analysis has other beneficial features, including theories for fitting models to data and for setting the optimal threshold, that we show could also be incorporated into CAP analysis. But the natural interpretation of the ROC measure of accuracy and the independence of ROC curves from the probability of default are advantages unavailable to CAP analysis.

Suggested Citation

  • Mr. R. John Irwin & Mr. Timothy C Irwin, 2012. "Appraising Credit Ratings: Does the CAP Fit Better than the ROC?," IMF Working Papers 2012/122, International Monetary Fund.
    • Handle: RePEc:imf:imfwpa:2012/122
    as

    Download full text from publisher

    File URL: http://www.imf.org/external/pubs/cat/longres.aspx?sk=25910
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Das, Sanjiv R. & Hanouna, Paul & Sarin, Atulya, 2009. "Accounting-based versus market-based cross-sectional models of CDS spreads," Journal of Banking & Finance, Elsevier, vol. 33(4), pages 719-730, April.
    2. Dirk Tasche, 2006. "Validation of internal rating systems and PD estimates," Papers physics/0606071, arXiv.org.
    3. Edward Altman & Gabriele Sabato, 2005. "Effects of the New Basel Capital Accord on Bank Capital Requirements for SMEs," Journal of Financial Services Research, Springer;Western Finance Association, vol. 28(1), pages 15-42, October.
    4. Miss Gabriela Dobrescu & Iva Petrova & Nazim Belhocine & Mr. Emanuele Baldacci, 2011. "Assessing Fiscal Stress," IMF Working Papers 2011/100, International Monetary Fund.
    5. Stein, Roger M., 2005. "The relationship between default prediction and lending profits: Integrating ROC analysis and loan pricing," Journal of Banking & Finance, Elsevier, vol. 29(5), pages 1213-1236, May.
    6. Crouhy, Michel & Galai, Dan & Mark, Robert, 2000. "A comparative analysis of current credit risk models," Journal of Banking & Finance, Elsevier, vol. 24(1-2), pages 59-117, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Turalay Kenc & Emrah Ismail Cevik & Sel Dibooglu, 2021. "Bank default indicators with volatility clustering," Annals of Finance, Springer, vol. 17(1), pages 127-151, March.
    2. Péter Bauer & Marianna Endrész, 2016. "Modelling Bankruptcy Using Hungarian Firm-Level Data," MNB Occasional Papers 2016/122, Magyar Nemzeti Bank (Central Bank of Hungary).
    3. Modina, Michele & Pietrovito, Filomena & Gallucci, Carmen & Formisano, Vincenzo, 2023. "Predicting SMEs’ default risk: Evidence from bank-firm relationship data," The Quarterly Review of Economics and Finance, Elsevier, vol. 89(C), pages 254-268.
    4. Nehrebecka Natalia, 2018. "An Evaluation of the Discriminatory Power of Selected Polish Bankruptcy Prediction Models As Part of the Validation Process," Financial Sciences. Nauki o Finansach, Sciendo, vol. 23(4), pages 63-88, December.
    5. Renatas Špicas & Airidas Neifaltas & Rasa Kanapickienė & Greta Keliuotytė-Staniulėnienė & Deimantė Vasiliauskaitė, 2023. "Estimating the Acceptance Probabilities of Consumer Loan Offers in an Online Loan Comparison and Brokerage Platform," Risks, MDPI, vol. 11(7), pages 1-30, July.
    6. Suleiman A. Badayi & Bolaji T. Matemilola & Bany‐Ariffin A.N & Lau Wei Theng, 2021. "Does corporate social responsibility influence firm probability of default?," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(3), pages 3377-3395, July.
    7. Pranith K. Roy & Krishnendu Shaw, 2023. "A credit scoring model for SMEs using AHP and TOPSIS," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(1), pages 372-391, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giulio Bottazzi & Marco Grazzi & Angelo Secchi & Federico Tamagni, 2011. "Financial and economic determinants of firm default," Journal of Evolutionary Economics, Springer, vol. 21(3), pages 373-406, August.
    2. Abinzano, Isabel & Gonzalez-Urteaga, Ana & Muga, Luis & Sanchez, Santiago, 2020. "Performance of default-risk measures: the sample matters," Journal of Banking & Finance, Elsevier, vol. 120(C).
    3. Correia, Maria & Kang, Johnny & Richardson, Scott, 2018. "Asset volatility," LSE Research Online Documents on Economics 84405, London School of Economics and Political Science, LSE Library.
    4. Francesco Dainelli & Francesco Giunta & Fabrizio Cipollini, 2013. "Determinants of SME credit worthiness under Basel rules: the value of credit history information," PSL Quarterly Review, Economia civile, vol. 66(264), pages 21-47.
    5. Raffaella Calabrese, 2012. "Improving Classifier Performance Assessment of Credit Scoring Models," Working Papers 201204, Geary Institute, University College Dublin.
    6. Maria Correia & Johnny Kang & Scott Richardson, 2018. "Asset volatility," Review of Accounting Studies, Springer, vol. 23(1), pages 37-94, March.
    7. Raffaella Calabrese, 2011. "Cost-sensitive classification for rare events: an application to the credit rating model validation for SMEs," Working Papers 201134, Geary Institute, University College Dublin.
    8. Suzan Hol, 2006. "The influence of the business cycle on bankruptcy probability," Discussion Papers 466, Statistics Norway, Research Department.
    9. Kern, Markus & Rudolph, Bernd, 2001. "Comparative analysis of alternative credit risk models: An application on German middle market loan portfolios," CFS Working Paper Series 2001/03, Center for Financial Studies (CFS).
    10. Pesaran, M. Hashem & Schuermann, Til & Treutler, Bjorn-Jakob & Weiner, Scott M., 2006. "Macroeconomic Dynamics and Credit Risk: A Global Perspective," Journal of Money, Credit and Banking, Blackwell Publishing, vol. 38(5), pages 1211-1261, August.
    11. Christian Gouriéroux & Alain Monfort, 2017. "Composite Indirect Inference with Application," Working Papers 2017-07, Center for Research in Economics and Statistics.
    12. Laura Jaramillo & Miss Anke Weber, 2013. "Global Spillovers into Domestic Bond Markets in Emerging Market Economies," IMF Working Papers 2013/264, International Monetary Fund.
    13. Hsin-Hui Chiu & Eva Wagner, 2020. "CEO Bonus Pay and Firm Credit Risk," International Journal of Risk and Contingency Management (IJRCM), IGI Global, vol. 9(1), pages 1-19, January.
    14. Honda, Jiro & Tapsoba, René & Issifou, Ismael, 2022. "When do we repair the roof? Insights from responses to fiscal crisis early warning signals," International Economics, Elsevier, vol. 172(C), pages 349-367.
    15. Marcin Chlebus, 2014. "One-day prediction of state of turbulence for financial instrument based on models for binary dependent variable," Ekonomia journal, Faculty of Economic Sciences, University of Warsaw, vol. 37.
    16. Kexue Liu & Jean Salvati & Mr. Renzo G Avesani & Mr. Alin T Mirestean, 2006. "Review and Implementation of Credit Risk Models of the Financial Sector Assessment Program (FSAP)," IMF Working Papers 2006/134, International Monetary Fund.
    17. Tang, Dragon Yongjun & Yan, Hong, 2017. "Understanding transactions prices in the credit default swaps market," Journal of Financial Markets, Elsevier, vol. 32(C), pages 1-27.
    18. Chiara Pederzoli & Grid Thoma & Costanza Torricelli, 2013. "Modelling Credit Risk for Innovative SMEs: the Role of Innovation Measures," Journal of Financial Services Research, Springer;Western Finance Association, vol. 44(1), pages 111-129, August.
    19. repec:onb:oenbwp:y:2002:i:3:b:3 is not listed on IDEAS
    20. Kanak Patel & Ricardo Pereira, 2007. "Expected Default Probabilities in Structural Models: Empirical Evidence," The Journal of Real Estate Finance and Economics, Springer, vol. 34(1), pages 107-133, January.
    21. Mikkel Hermansen & Oliver Röhn, 2017. "Economic resilience: The usefulness of early warning indicators in OECD countries," OECD Journal: Economic Studies, OECD Publishing, vol. 2016(1), pages 9-35.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:imf:imfwpa:2012/122. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Akshay Modi (email available below). General contact details of provider: https://edirc.repec.org/data/imfffus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.