A Multilayer Approach of Anomaly Detection for Email Systems

Many techniques have been applied to anomaly detection to detect novel attacks, such as statistical analysis, clustering, support vector machines, neural networks and etc. Although the results are promising, there’s still a serious problem, high false positive rates, which make anomaly detection systems practically unusable. We observe that most network Intrusion Detection systems (IDSs) work on information that is only available on lower layers of the network or on higher layers, but not on both. We argue that by correlating the information on different layers, we can have a more efficient anomaly detection system. We introduce an anomaly detection system based on the layer correlation. Bayesian networks and statistical analysis are used to build normal system models for the anomaly detection engine. The prototype system is tested on tcpdump traces including normal and anomalous email activities. Our experimental results show that our proposed solution is capable of reducing false alarm rates.