DATALITE: a distributed architecture for traffic analysis via light-weight traffic digest

In this paper, we propose DATALITE, a Distributed Architecture for Traffic Analysis via LIght-weight Traffic digEst, which introduces a set of new distributed algorithms and protocols to support general Traffic Measurement and Analysis (TMA) functions for large-scale, 10Gbps+ packet-switched networks. We formulate the network-wide traffic measurement/ analysis problem as a series of set-cardinality-determination (SCD) problems. By leveraging recent advances in probabilistic distinct sample counting techniques, the set-cardinalities, and thus, the network-wide traffic measurements of interest can be computed in a distributed manner via the exchange of extremely light-weight traffic digests (TD’s) amongst the network nodes. A TD for N packets only requires O(loglog N) bits of memory storage.