Fast String Matching with Overlapped Substring Classifier in Deep Packet Inspection Systems

Traditional DFA based DPI (Deep Packet Inspection) string matching architectures either suffer from throughput bottleneck or unfeasible memory requirement, or both. Bloom Filter based schemes, on the other hand, only provide indefinite and unprecise match results. In this paper, we propose a novel string matching data structure called Overlapped Substring Classifier(OSC), which tries to compromise between these two ends. Instead of using incoming byte flow directly, we use OSC to extract the characteristic digest of the incoming string, which we demonstrate would be sufficient for locating a very small set of possible match, using DFA techniques. This type of match ambiguity and false-positive inaccuracy can be tuned to be negligible. The scheme is perfectly suitable for efficient and parallel hardware implementation, which makes ultra high performance and low memory usage simultaneously possible. A hardware architecture is also designed supporting single-threaded scanning rate of 10Gbp, with only moderate memory requirement and clock rate assumption.