Capturing the Elusive Poissonity in Web Traffic

Numerous studies have shown that the process of packet arrivals from Web traffic exhibits strong long-range dependence, which makes it not amenable to be described using the convenient but necessarily short-range dependent framework of Poisson modeling. However, Web traffic is ultimately driven by independent human behavior, so it seems natural to search for an underlying "seed process", consistent with Poissonity, indirectly driving the packet arrivals of Web traffic. Our study examines Web traffic at different levels of packet aggregation, using powerful statistical analysis tools for identifying the finest level that can be effectively modeled using a homogeneous Poisson process. We show that the arrivals of HTTP responses, TCP connections and Web pages do not provide a satisfactory seed process. However, we find Poissonity in the arrivals of "navigation bursts". A navigation burst is a tightly-spaced sequence of Web pages downloaded by the same Web client, which can be explained by fast navigation through several pages before reaching relevant content. Our analysis suggests that the start times of such navigation bursts, which we identify by detecting user think times between 12 and 30 seconds, can be effectively modeled as a homogeneous Poisson process. We believe that our methodology can be extended to other complex modeling problems where finding Poissonity can greatly simplify parsimonious modeling.