Efficient statistics gathering from tree-search methods in packet processing systems

We present a novel algorithm for efficiently gathering statistics about the hit frequencies on the nodes of a search tree in a packet processing system, under limiting space constraints. The expand and collapse (EaC) algorithm is a heuristic that periodically adjusts the subset of nodes of the search tree at which statistics are gathered, in order to use the limited space available to collect statistics in preference from the currently most heavily-hit nodes in the search tree. We prove convergence and good node-hit coverage of the algorithm and validate its performance on a set of simulated data. The information collected can be useful for a variety of reasons, such as inferring traffic properties, discovering failures and attacks or dynamically optimizing the search method itself for locality patterns in the oncoming traffic.