MT-FBERT: Malicious Traffic Detection Based on Efficient Federated Learning of BERT

The rising frequency of network intrusions has significantly impacted critical infrastructures, leading to an increased focus on the detection of malicious network traffic in recent years. However, traditional port-based and classical machine learning-based malicious network traffic detection methods suffer from a dependence on expert experience and limited generalizability. In this paper, we propose a malicious traffic detection method based on an efficient federated learning framework of Bidirectional Encoder Representations from Transformers (BERT), called MT-FBERT. It offers two major advantages over most existing approaches. First, MT-FBERT pretrains BERT using two pre-training tasks along with an overall pre-training loss on large-scale unlabeled network traffic, allowing the model to automatically learn generalized traffic representations, which do not require human experience to extract the behavior features or label the malicious samples. Second, MT-FBERT finetunes BERT for malicious network traffic detection through an efficient federated learning framework, which both protects the data privacy of critical infrastructures and reduces resource consumption by dynamically identifying and updating only the most significant neurons in the global model. Evaluation experiments on public datasets demonstrated that MT-FBERT outperforms state-of-the-art baselines in malicious network traffic detection.