Efficient Anomaly Detection for Cyber-Physical Leveraging Knowledge Distillation and Model Quantization

Cyber-Physical Systems (CPS) are fundamental to modern technological advancement, they integrate physical and computational processes enabling real time interactions, and are often under constant security threats. Intrusion Detection Systems are essential in this context, they continuously monitor network traffic for anomalies that could indicate cyber-attacks. However, CPS are usually resource constrained environments and IDS can be computationally demanding, which makes real-time intrusion challenging. In this work, we address these limitations by providing a lightweight intrusion detection combining feature selection, knowledge distillation and quantization, aiming to drastically reduce the intrusion detection model size and inference time. In this setup, we experimented with 10, 20, 30 and 40 features from the NSL-KDD dataset, and developed a teacher model, a student model and a quantized model for each set of features. As a result, we found that the 30 features configuration set achieves the best balance between performance and efficiency, where the teacher model achieves 98.91% accuracy at 627 KB, the distilled model with 98.78% accuracy at 39.9 KB and finally, the quantized model achieves 98.80% accuracy at only 7.96 KB. These results showcase the abilities of combining these techniques in achieving low-cost intrusion detection without compromising performance.