Intelligent Flow-Based Sampling for Effective Network Anomaly Detection

Sampling has become an essential component of scalable Internet traffic monitoring and anomaly detection. In this paper, the emphasis is placed on the evaluation of the impact of using intelligent flow sampling techniques on the anomaly detection process. Based on the observation that small flows are usually the source of many network attacks (DDoS, portscans, worm propagation) we first introduce a new flow sampling methodology that focuses on the selection of small flows and achieves to improve anomaly detection effectiveness, while at the same time reduces the number of selected flows. The performance evaluation of the impact of intelligent flow-based sampling on the anomaly detection process is achieved through the adoption and application of a sequential non-parametric Change-Point Detection anomaly detection method on realistic data that have been collected from a real operational university campus network.