Mouse Trapping: A Flow Data Reduction Method

Flow based traffic measurement today is a very important tool for network management but suffers from huge amounts of data and a lack of scalability. Therefore it is important to find methods to reduce that amount of data for applications like long-term archiving or filtering in mediators to improve scalability. A fact that helps here is, that general internet traffic has power-law characteristics and that for many applications it is enough to only look at the large flows. In this work we introduce Mouse Trapping, a flow data reduction method that keeps the large flow records, while the small flow records are aggregated or even removed. We show based on theoretical simulation, that because of the heavy-tail nature of normal flow data, the main part of the traffic is represented by only a few large flow records, while the small flow records represent only a small part of the traffic. In an evaluation with real traffic data we can confirm that the traffic flows are in fact mostly power-law distributed. We can show that with this method, flow data can be reduced by up to 90% if all small flow records are just discarded, affecting only flow records of 5% of the traffic.