Real time multi stage unsupervised intelligent engine for NIDS to enhance detection rate of unknown attacks

The most traditional technique for Network Intrusion Detection Systems (NIDSs) is misuse detection which only detects well-known attacks by matching the current behavior of network with pre-defined attacks´ signatures. Providing attacks´ signatures is costly, time consuming and with the explosive growing number of zero day attacks, using misuse detection mechanism is not an efficient solution. Other techniques which applied on NIDS are supervised and semi-supervised anomaly detection systems which can detect novel attacks by comparing the current behavior of the network to the training sample; however producing labeled or attack-free dataset is difficult for training the engine. Current NIDS solutions monitor bytes, packets´ payload or network flows to detect intrusions. Today it is difficult to monitor the payload of packets in high speed network (1-10 Gbps) and recent network attacks are becoming more complex and analyzing only the payload of packets will not produce enough information for detection engine. In this paper we propose a new Real Time Unsupervised Network Intrusion Detection System (RTUNIDS) which monitor network flows in two windows with different sizes and detects network attacks by correlating outliers from multiple clusters. The proposed solution has the ability of detecting different types of intrusions in realtime such as DOS, DDOS, scanning, distribution of worms and any other network attacks which produce huge amount of network traffic and in the meanwhile it detects Bot-Master if the detected attack lunched by Bots.