Detecting denial-of-service attacks with incomplete audit data

With the ever increasing deployment and usage of gigabit networks, traditional network anomaly detection based intrusion detection systems have not scaled accordingly. Most, if not all, systems deployed assume the availability of complete and clean data for the purpose of intrusion detection. We contend that this assumption is not valid. Factors like noise in the audit data, mobility of the nodes and the large amount of network data generated by the network make it difficult to build a normal traffic profile of the network for the purpose of anomaly detection. From this perspective, we present an anomaly detection scheme, called SCAN (stochastic clustering algorithm for network anomaly detection), that has the capability to detect intrusions with high accuracy even when audit data is not complete. We use the expectation-maximization algorithm to cluster the incoming audit data and compute the missing values in the audit data. We improve the speed of convergence of the clustering process by using Bloom filters and data summaries. We evaluate SCAN using the 1999 DARPA/Lincoln Laboratory intrusion detection evaluation dataset.