Distributed Data Collection, Correlation and Data Reduction for Iterative Streaming Analytics in Wireless Mobile Networks

In this paper, we propose a policy-based, real-time packet analytics framework developed to evaluate 3G and 4G mobile communication technologies, and the associated signaling, tunneling, & user application protocols with the ultimate goal of providing context-enriched correlated information to streaming analytics platforms based on business needs. The framework allows user-defined dimensioning, telemetry generation and Key Performance Indicators (KPI) calculations with a focus on real-time network event measurements on the order of milliseconds to seconds. Unlike traditional streaming analytics platforms which rely on raw data logs and call detail records (CDRs), our framework allows augmenting data streams, such as binding TCP, HTTP, or application-level transactions with pre-correlated network state, subscriber patterns, location changes, service indicators, and computed KPIs, for continuous export into business intelligence architectures. By way of example, we present in detail, a closed-loop network optimization use case constituting a stream of KPIs including cell utilization indicators, subscriber mobility behavior, and application usage patterns towards a Business Intelligence (BI) policy enforcement point. This facilitates real-time actions such as congestion mitigation, and QoE management across heterogeneous networks and application behaviors.