Bayesian probability of default models with Langevin dynamics

Using machine learning to estimate the probability of default in credit risk is becoming a popular approach. Bayesian methodologies offer a probabilistic interpretation of model predictions and prevent overfitting, a significant flaw in numerous machine learning models. However, Bayesian inference based on Monte Carlo Markov Chain (MCMC) algorithms comes with high computational costs. For credit scoring models efficiency and performance are equally important features. We propose two machine learning architectures based on Stochastic Gradient Langevin Dynamics (SGLD) to estimate the probability of default of loan applicants. This framework (i) allows us to sample from the true posterior without relying on typical MCMC algorithms, (ii) it is not computationally expensive and (iii) it leverages the strength of Bayesian approaches, such as the flexibility to regularization. We apply this method to Bayesian Logistic Regression and Bayesian Neural Network. Furthermore, we perform a benchmarking analysis with different models and regularization techniques on four large retail loan datasets. We also address model explainability with the model-agnostic method of Shapley Additive Explanation (SHAP).