An extensive experimental comparison of machine and deep learning methods for credit and bank fraud detection

The growing prevalence of financial fraud in the digital era has motivated the research community to develop a variety of fraud detection methods and create benchmark datasets. However, many studies analyse these datasets in isolation, make comparisons with a few existing methods and evaluate performance using selected metrics. To address this gap, we present an extensive experimental comparison of 9 methods over 9 fraud-related datasets using 9 evaluation metrics. This allows us to determine the best-performing method and to make recommendations that are independent of the specific dataset or evaluation metrics used. To achieve this, we use the corrected Friedman test to determine whether there is a statistically significant difference in performance of the methods over all the datasets for a given evaluation metric. Subsequently, the Nemenyi post-hoc test and critical difference – or the minimum difference in average ranks of two methods for the performance to be significant – are used to assign a score to each method. The scores are then aggregated across all the evaluation metrics to determine the method with the highest overall performance. XGBoost and random forest showed the best performance across all the evaluation metrics followed by AutoKeras. However, at the business-imposed limit of 5% false positive rate (FPR), the state-of-the-art convolutional spiking neural networks (CSNN) had the best FPR ratio of young and old customers, and recall. Based on these findings, we recommend that when new fraud detection methods are proposed, they should be compared with XGBoost, RF and CSNN using a broad range of evaluation metrics.