IDEAS home Printed from https://ideas.repec.org/p/bcb/wpaper/582.html
   My bibliography  Save this paper

A Novel Credit Model Risk Measure: does more data lead to lower model risk in credit scoring models?

Author

Listed:
  • Valter T. Yoshida Jr
  • Alan de Genaro
  • Rafael Schiozer
  • Toni R. E. dos Santos

Abstract

Large databases and Machine Learning have increased our ability to produce models with a different number of observations and explanatory variables. The credit scoring literature has focused on the optimization of classifications. Little attention has been paid to the inadequate use of models. This study fills this gap by focusing on model risk. It proposes a measure to assess credit scoring model risk. Its emphasis is on model misuse. The proposed model risk measure is ordinal, and it applies to many settings and types of loan portfolios, allowing comparisons of different specifications and situations (as in-sample or out-of-sample data). It allows practitioners and regulators to evaluate and compare different credit risk models in terms of model risk. We empirically test our measure in plugin LASSO default models and find that adding loans from different banks to increase the number of observations is not optimal, challenging the generally accepted assumption that more data leads to better predictions.

Suggested Citation

  • Valter T. Yoshida Jr & Alan de Genaro & Rafael Schiozer & Toni R. E. dos Santos, 2023. "A Novel Credit Model Risk Measure: does more data lead to lower model risk in credit scoring models?," Working Papers Series 582, Central Bank of Brazil, Research Department.
  • Handle: RePEc:bcb:wpaper:582
    as

    Download full text from publisher

    File URL: https://www.bcb.gov.br/content/publicacoes/WorkingPaperSeries/WP582.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bernardus Van Doornik & Dimas Fazio & David Schoenherr & Janis Skrastins, 2022. "Unemployment Insurance as a Subsidy to Risky Firms," The Review of Financial Studies, Society for Financial Studies, vol. 35(12), pages 5535-5595.
    2. Bernard, Carole & Vanduffel, Steven, 2015. "A new approach to assessing model risk in high dimensions," Journal of Banking & Finance, Elsevier, vol. 58(C), pages 166-178.
    3. Coqueret, Guillaume & Tavin, Bertrand, 2016. "An investigation of model risk in a market with jumps and stochastic volatility," European Journal of Operational Research, Elsevier, vol. 253(3), pages 648-658.
    4. Cosmin L. Ilut & Martin Schneider, 2022. "Modeling Uncertainty as Ambiguity: a Review," NBER Working Papers 29915, National Bureau of Economic Research, Inc.
    5. Rama Cont, 2006. "Model Uncertainty And Its Impact On The Pricing Of Derivative Instruments," Mathematical Finance, Wiley Blackwell, vol. 16(3), pages 519-547, July.
    6. Yiping Huang & Ms. Longmei Zhang & Zhenhua Li & Han Qiu & Tao Sun & Xue Wang, 2020. "Fintech Credit Risk Assessment for SMEs: Evidence from China," IMF Working Papers 2020/193, International Monetary Fund.
    7. Andrés Alonso & José Manuel Carbó, 2020. "Machine learning in credit risk: measuring the dilemma between prediction and supervisory cost," Working Papers 2032, Banco de España.
    8. Ceylan Onay & Elif Öztürk, 2018. "A review of credit scoring research in the age of Big Data," Journal of Financial Regulation and Compliance, Emerald Group Publishing Limited, vol. 26(3), pages 382-405, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Valeriane Jokhadze & Wolfgang M. Schmidt, 2020. "Measuring Model Risk In Financial Risk Management And Pricing," International Journal of Theoretical and Applied Finance (IJTAF), World Scientific Publishing Co. Pte. Ltd., vol. 23(02), pages 1-37, April.
    2. Fritzsch, Simon & Timphus, Maike & Weiß, Gregor, 2024. "Marginals versus copulas: Which account for more model risk in multivariate risk forecasting?," Journal of Banking & Finance, Elsevier, vol. 158(C).
    3. Baule, Rainer & Shkel, David, 2021. "Model risk and model choice in the case of barrier options and bonus certificates," Journal of Banking & Finance, Elsevier, vol. 133(C).
    4. Marcelo Brutti Righi, 2018. "A theory for combinations of risk measures," Papers 1807.01977, arXiv.org, revised May 2023.
    5. Lazar, Emese & Qi, Shuyuan, 2022. "Model risk in the over-the-counter market," European Journal of Operational Research, Elsevier, vol. 298(2), pages 769-784.
    6. Righi, Marcelo Brutti & Müller, Fernanda Maria & Moresco, Marlon Ruoso, 2020. "On a robust risk measurement approach for capital determination errors minimization," Insurance: Mathematics and Economics, Elsevier, vol. 95(C), pages 199-211.
    7. Emese Lazar & Shuyuan Qi & Radu Tunaru, 2020. "Measures of Model Risk in Continuous-time Finance Models," Papers 2010.08113, arXiv.org, revised Oct 2020.
    8. Andrés Alonso & José Manuel Carbó, 2021. "Understanding the performance of machine learning models to predict credit default: a novel approach for supervisory evaluation," Working Papers 2105, Banco de España.
    9. Mark Cummins & Fabian Gogolin & Fearghal Kearney & Greg Kiely & Bernard Murphy, 2023. "Practice-relevant model validation: distributional parameter risk analysis in financial model risk management," Annals of Operations Research, Springer, vol. 330(1), pages 431-455, November.
    10. Simon Fritzsch & Maike Timphus & Gregor Weiss, 2021. "Marginals Versus Copulas: Which Account For More Model Risk In Multivariate Risk Forecasting?," Papers 2109.10946, arXiv.org.
    11. G. Rigatos & N. Zervos, 2017. "Detection of Mispricing in the Black–Scholes PDE Using the Derivative-Free Nonlinear Kalman Filter," Computational Economics, Springer;Society for Computational Economics, vol. 50(1), pages 1-20, June.
    12. Ankush Agarwal & Stefano de Marco & Emmanuel Gobet & Gang Liu, 2017. "Rare event simulation related to financial risks: efficient estimation and sensitivity analysis," Working Papers hal-01219616, HAL.
    13. Farkas, Walter & Fringuellotti, Fulvia & Tunaru, Radu, 2020. "A cost-benefit analysis of capital requirements adjusted for model risk," Journal of Corporate Finance, Elsevier, vol. 65(C).
    14. Wosnitza, Jan Henrik, 2022. "Calibration alternatives to logistic regression and their potential for transferring the dispersion of discriminatory power into uncertainties of probabilities of default," Discussion Papers 04/2022, Deutsche Bundesbank.
    15. Christophe Boucher & Benjamin Hamidi & Patrick Kouontchou & Bertrand Maillet, 2012. "Une évaluation économique du risque de modèle pour les investisseurs de long terme," Revue économique, Presses de Sciences-Po, vol. 63(3), pages 591-600.
    16. Carole Bernard & Oleg Bondarenko & Steven Vanduffel, 2018. "Rearrangement algorithm and maximum entropy," Annals of Operations Research, Springer, vol. 261(1), pages 107-134, February.
    17. Pedro Guerra & Mauro Castelli, 2021. "Machine Learning Applied to Banking Supervision a Literature Review," Risks, MDPI, vol. 9(7), pages 1-24, July.
    18. Ben R. Craig & Margherita Giuzio & Sandra Paterlini, 2019. "The Effect of Possible EU Diversification Requirements on the Risk of Banks’ Sovereign Bond Portfolios," Working Papers 19-12, Federal Reserve Bank of Cleveland.
    19. Thibaut Lux & Antonis Papapantoleon, 2016. "Model-free bounds on Value-at-Risk using extreme value information and statistical distances," Papers 1610.09734, arXiv.org, revised Nov 2018.
    20. Pamela Giustinelli, 2022. "Expectations in Education: Framework, Elicitation, and Evidence," Working Papers 2022-026, Human Capital and Economic Opportunity Working Group.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bcb:wpaper:582. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Rodrigo Barbone Gonzalez (email available below). General contact details of provider: https://www.bcb.gov.br/en .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.