Persistent dataset generation using real-time operative framework

During the widening of information technology, the need to a framework that efficiently constructs connection vectors from online data flow for evaluating intrusion detection models has become fundamental. Moreover, known datasets in intrusion detection are either outdated or offline aggregated. Therefore, these datasets are not adequate for performance evaluation anymore. In this paper we present a novel framework, OptiFilter, that mines network packets and host events, based on significant features in intrusion detection. The framework collects network packets and host events continuously in real-time and parses them to a queue of dynamic windows, then it generates connection vectors accordingly. We evaluate the framework in a real-time heterogeneous network and compare it with other benchmark datasets. Our framework shows promising results with minimal processing time for maximum amount of packets. Moreover, it can constantly produce significant and meaningful datasets for evaluating intrusion detection systems.