A Two-Stage Process Based on Data Mining and Optimization to Identify False Positives and False Negatives Generated by Intrusion Detection Systems

To ensure the protection of computer networks, an intrusion detection system (IDS) should be integrated in the security infrastructure. However, IDSs generate a high amount of false alerts exceeding the administrator ability for analysis and omit several attacks which can threaten the network security. In this paper, a two-stage process based on data mining and optimization is proposed having as input the outcome of multiple IDSs. In the first stage, for each IDS the set of elementary alerts is clustered to create a set of meta-alerts. Then, we remove false positives from the sets of meta-alerts using a binary optimization problem. In the second stage, we discard the meta-alerts generated by all IDSs and only those missed by one, two or most of them are left. This set is called the set of potential false negatives. In fact, at this level a meta-alerts fusion is performed to avoid the redundancy between meta-alerts collected from multiple IDSs. Finally, a binary classification algorithm is proposed to classify the potential false negatives either as real attacks or not. Experimental results show that our proposed process outperforms concurrent methods by significantly reducing the rate of false positives and false negatives.