Attention and residual mechanism-based CNN architecture (ARC-Net) with enhanced fairness generalization for deepfake facial image detection

Deepfake (DF) content poses a major challenge to digital media authentication, that can mimic facial movements, creating realistic replicas that risk spreading misinformation and enabling harassment, as can be seen in Bangladesh. Previous studies applied residual convolutional blocks or attention mechanisms for DF detection; however, these approaches often treat these components in isolation and lack explicit consideration of fairness, out-of-distribution generalization, statistical analysis or artifact-focused detection. Our study introduces an approach, called ARC-Net, which uses a combination of attention and residual convolutional layers along with the EfficientNet B0 base, the attention mechanism of which allows the model to pay more attention to details that could resemble the ones seen in non-perfectly executed DF, which enhances the model’s potential to distinguish them, addressing limitations in pre-trained and also advanced DL models. A dataset of 500 real images from Bangladesh combined with a 140k real and fake faces dataset was used to train and test our model alongside four pre-trained DL models. ARC-Net performed much better than the other traditional and state of art methods with 99% accuracy, 1.0 precision, 0.97 recall and 0.98 F1 score, reaching the highest level of reliability in spotting DF images. To assess the external reliability and generalizability of ARC-Net, the model was evaluated on the Deepfake Dataset and the Deepfake Database datasets, achieving consistently high and balanced performance across different scales. Three out-of-distribution (OOD) experiments were conducted, where the first one evaluated South Asian images, showing that incorporating fewer than one percent real Bangladeshi images reduced the false positive rate by more than half and improved probability calibration, while the remaining two cross-dataset experiments demonstrated strong transferability. An ablation study further showed the impact of different components within a model by systematically removing or modifying them and statistical significance between competing classifiers was assessed using McNemar’s Statistical test. In addition, we’ve applied Explainable AI (XAI) techniques, Grad-CAM and LIME to offer transparency of the results, as giving attention to the facial region is also important for DF detection. This study will help advance DF detection by integrating ARC-Net’s attention-residual mechanisms and XAI, offering insights for developing models in security and media forensics.