Scalable Pattern-Matching via Dynamic Differentiated Distributed Detection (D4)

Pattern Matching (PM) over network packet flows for Network Intrusion Detection/Prevention System is becoming more and more performance sensitive due to the rapid progress of Internet applications in terms of data volumes. Meanwhile, modern multicore platforms are becoming performance competitive with traditional hardware solutions for PM. But due to the unbalance of network flow sizes, traditional flow- based data parallel processing/programming model can not fully exert multicore platforms\´ computing power and results in poor performance scalability. In this paper, a novel parallel inspection model, Dynamic Differentiated Distributed Detection (D⁴) is proposed. D⁴ deploys distributed parallel operations by adding one more dimension on workload partition/allocation. It proposes an effective and efficient scheme to pre-partition the pattern set in several candidate ways, called "Detection Modes", and let multiple candidate PM methods to handle the subsets, respectively; the most suitable Detection Mode would be selected specifically for each incoming flows at the run-time, and the workload would be dynamically allocated among multiple CPU cores. Experimental results on real-world pattern set and traffic traces show that D⁴ scales much better than traditional schemes by better balancing the load among the processors while avoiding unnecessary overheads.