Counting Evolving Data Stream Based on Hierarchical Counting Bloom Filter

In many data stream oriented application circumstances, frequency distribution of elements meets heavy-tailed distribution, which means most elements have small frequencies and few elements have high frequencies. Consequently, traditional counting bloom filters (CBF) and dynamic count filters (DCF) cannot represent the data effectively. In order to improve the data processing efficiency, this paper proposes a novel hierarchical counting bloom filters (HCBF) data structure. We evaluate the performance of HCBF by theoretical analysis and simulation experiments. Both analysis and simulation results show that HCBF significantly reduces space complexity while maintaining better time complexity and error rate according to CBF and DCF.