Dealing with Dataset Class Imbalance for Multi-Class Network Intrusion Detection Systems

Intrusion Detection Systems (IDS) are now more crucial for protecting network environments due to the increasing number of IoT devices. However, class imbalance substantially impacts the performance of IDS, especially in multi-class classification, where some attack classes are prevalent, and others are rare. Our research explores the effects of class imbalance by experimenting with the CICIoT2023 dataset in three distinct classification settings: 2-class, 8-class, and 33-class. Our experiments reveal that, although the binary classification performance is high (accuracy ≈ 0.99), the performance drops in multi-class settings, with accuracy of ≈ 0.85 and macro-F1 of 0.59 in the 8-class scenario, and further drops to accuracy of ≈ 0.76 and macro-F1 of 0.47 in the 33-class setting due to the severe class imbalance. To resolve this problem, we examine SMOTE, class weighting, and a combined SMOTE-ENN method. Our results show that the SMOTE-ENN approach effectively balances classes (up to 0.97 accuracy and macro-F1≈0.96 on the test dataset after training with balanced data) and enhances detection performance for minority classes. By contrast, baseline models have lower macro-F1 and recall for minority classes. These results show that hybrid resampling not only boosts classification accuracy but also the ability to detect rare attacks, thus serving as an effective strategy for robust multi-class IoT intrusion detection systems.