Detecting Persistent User Behavior Using Probabilistic Counting in Network-Wide View

Persistent user behavior monitoring, which deals with finding users that occur persistently over a measurement period, is one hot topic in traffic measurement. It is significant for many applications, such as anomaly detection. Former works concentrate on monitoring frequent user behavior, such as users occurring frequently either over one measurement period or on one monitor. They have paid little attention to detect persistent user behavior over a long measurement period on multiple monitors. However, persistent users do not necessarily appear frequently in a short measurement period, but appear persistently in a long measurement period. Due to limited resource on monitors, it is not practical to collect a tremendous amount of network traffic in a long measurement period on one single monitor. Moreover, since network attackers deliberately send packets flowing through the entire managed network, it is difficult to detect abnormal behavior on one single monitor. To solve the above challenges, a novel method for detecting persistent user behavior called DPU is proposed, and it contains both online distributed traffic collection in a long measurement period on multiple monitors and offline centralized user behavior detection on the central server. The key idea of DPU is that we design the compact distributed synopsis data structure to collect the relevant information with users occurring in a long measurement period on each monitor, and we can reconstruct user IDs using simple calculations and bit settings to find users with persistent behavior on the basis of estimated occurrence frequency of users on the central server when user IDs are unknown in advance. The experiments are conducted on real traffic to evaluate the performance of detecting persistent user behavior, and the experimental results illustrate that our method can improve about 30% estimation accuracy, 40% detection precision, and accelerate about 3 times in comparison with the related method.