IDEAS home Printed from https://ideas.repec.org/p/bcb/wpaper/582.html
   My bibliography  Save this paper

A Novel Credit Model Risk Measure: does more data lead to lower model risk in credit scoring models?

Author

Listed:
  • Valter T. Yoshida Jr
  • Alan de Genaro
  • Rafael Schiozer
  • Toni R. E. dos Santos

Abstract

Large databases and Machine Learning have increased our ability to produce models with a different number of observations and explanatory variables. The credit scoring literature has focused on the optimization of classifications. Little attention has been paid to the inadequate use of models. This study fills this gap by focusing on model risk. It proposes a measure to assess credit scoring model risk. Its emphasis is on model misuse. The proposed model risk measure is ordinal, and it applies to many settings and types of loan portfolios, allowing comparisons of different specifications and situations (as in-sample or out-of-sample data). It allows practitioners and regulators to evaluate and compare different credit risk models in terms of model risk. We empirically test our measure in plugin LASSO default models and find that adding loans from different banks to increase the number of observations is not optimal, challenging the generally accepted assumption that more data leads to better predictions.

Suggested Citation

  • Valter T. Yoshida Jr & Alan de Genaro & Rafael Schiozer & Toni R. E. dos Santos, 2023. "A Novel Credit Model Risk Measure: does more data lead to lower model risk in credit scoring models?," Working Papers Series 582, Central Bank of Brazil, Research Department.
  • Handle: RePEc:bcb:wpaper:582
    as

    Download full text from publisher

    File URL: https://www.bcb.gov.br/content/publicacoes/WorkingPaperSeries/WP582.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bernardus Van Doornik & Dimas Fazio & David Schoenherr & Janis Skrastins, 2022. "Unemployment Insurance as a Subsidy to Risky Firms," The Review of Financial Studies, Society for Financial Studies, vol. 35(12), pages 5535-5595.
    2. Yiping Huang & Ms. Longmei Zhang & Zhenhua Li & Han Qiu & Tao Sun & Xue Wang, 2020. "Fintech Credit Risk Assessment for SMEs: Evidence from China," IMF Working Papers 2020/193, International Monetary Fund.
    3. Cosmin L. Ilut & Martin Schneider, 2022. "Modeling Uncertainty as Ambiguity: a Review," NBER Working Papers 29915, National Bureau of Economic Research, Inc.
    4. Rama Cont, 2006. "Model Uncertainty And Its Impact On The Pricing Of Derivative Instruments," Mathematical Finance, Wiley Blackwell, vol. 16(3), pages 519-547, July.
    5. Andrés Alonso & José Manuel Carbó, 2020. "Machine learning in credit risk: measuring the dilemma between prediction and supervisory cost," Working Papers 2032, Banco de España.
    6. Ceylan Onay & Elif Öztürk, 2018. "A review of credit scoring research in the age of Big Data," Journal of Financial Regulation and Compliance, Emerald Group Publishing Limited, vol. 26(3), pages 382-405, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Andrés Alonso & José Manuel Carbó, 2021. "Understanding the performance of machine learning models to predict credit default: a novel approach for supervisory evaluation," Working Papers 2105, Banco de España.
    2. Wosnitza, Jan Henrik, 2022. "Calibration alternatives to logistic regression and their potential for transferring the dispersion of discriminatory power into uncertainties of probabilities of default," Discussion Papers 04/2022, Deutsche Bundesbank.
    3. Christophe Boucher & Benjamin Hamidi & Patrick Kouontchou & Bertrand Maillet, 2012. "Une évaluation économique du risque de modèle pour les investisseurs de long terme," Revue économique, Presses de Sciences-Po, vol. 63(3), pages 591-600.
    4. Faraz Ahmed & Kehkashan Nizam & Zubair Sajid & Sunain Qamar & Ahsan, 2024. "Striking a Balance: Evaluating Credit Risk with Traditional and Machine Learning Models," Bulletin of Business and Economics (BBE), Research Foundation for Humanity (RFH), vol. 13(3), pages 30-35.
    5. Barrieu, Pauline & Scandolo, Giacomo, 2014. "Assessing financial model risk," LSE Research Online Documents on Economics 60084, London School of Economics and Political Science, LSE Library.
    6. Carol Alexander & José María Sarabia, 2012. "Quantile Uncertainty and Value‐at‐Risk Model Risk," Risk Analysis, John Wiley & Sons, vol. 32(8), pages 1293-1308, August.
    7. Lazar, Emese & Zhang, Ning, 2019. "Model risk of expected shortfall," Journal of Banking & Finance, Elsevier, vol. 105(C), pages 74-93.
    8. Patrick Beissner, 2019. "Coherent-Price Systems and Uncertainty-Neutral Valuation," Risks, MDPI, vol. 7(3), pages 1-18, September.
    9. Mai Jan-Frederik & Schenk Steffen & Scherer Matthias, 2015. "Analyzing model robustness via a distortion of the stochastic root: A Dirichlet prior approach," Statistics & Risk Modeling, De Gruyter, vol. 32(3-4), pages 177-195, December.
    10. Virmani, Vineet, 2014. "Model Risk in Pricing Path-dependent Derivatives: An Illustration," IIMA Working Papers WP2014-03-22, Indian Institute of Management Ahmedabad, Research and Publication Department.
    11. Marcel Nutz, 2013. "Utility Maximization under Model Uncertainty in Discrete Time," Papers 1307.3597, arXiv.org.
    12. Pagano, Marco & Picariello, Luca, 2023. "Talent discovery, layoff risk and unemployment insurance," European Economic Review, Elsevier, vol. 154(C).
    13. Kamila Sommer, 2014. "Fertility Choice in a Life Cycle Model with Idiosyncratic Uninsurable Earnings Risk," Finance and Economics Discussion Series 2014-32, Board of Governors of the Federal Reserve System (U.S.).
    14. Asimit, Alexandru V. & Bignozzi, Valeria & Cheung, Ka Chun & Hu, Junlei & Kim, Eun-Seok, 2017. "Robust and Pareto optimality of insurance contracts," European Journal of Operational Research, Elsevier, vol. 262(2), pages 720-732.
    15. Carol Alexander & Jose Maria Sarabia, 2010. "Endogenizing Model Risk to Quantile Estimates," ICMA Centre Discussion Papers in Finance icma-dp2010-07, Henley Business School, University of Reading.
    16. Siu, Tak Kuen, 2023. "European option pricing with market frictions, regime switches and model uncertainty," Insurance: Mathematics and Economics, Elsevier, vol. 113(C), pages 233-250.
    17. Danielsson, Jon & James, Kevin R. & Valenzuela, Marcela & Zer, Ilknur, 2016. "Model risk of risk models," Journal of Financial Stability, Elsevier, vol. 23(C), pages 79-91.
    18. Zixue Zhao & Tianxiang Cui & Shusheng Ding & Jiawei Li & Anthony Graham Bellotti, 2024. "Resampling Techniques Study on Class Imbalance Problem in Credit Risk Prediction," Mathematics, MDPI, vol. 12(5), pages 1-27, February.
    19. Johannes Siven & Rolf Poulsen, 2009. "Auto-static for the people: risk-minimizing hedges of barrier options," Review of Derivatives Research, Springer, vol. 12(3), pages 193-211, October.
    20. Jakob Sohl, 2012. "Confidence sets in nonparametric calibration of exponential L\'evy models," Papers 1202.6611, arXiv.org, revised Sep 2013.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bcb:wpaper:582. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Rodrigo Barbone Gonzalez (email available below). General contact details of provider: https://www.bcb.gov.br/en .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.