Resource Management Considerations in Collector Overlay Networks

Streaming monitorables on an overlay plane orthogonal to the data-path is emerging as a promising way to monitor, detect and isolate network impairments and outages. However, keeping the overlay stable and efficient poses acute challenges, since the overlay network itself shall not be monitored for outages. A linear mapping of exporter elements onto collectors often leads to a sub-optimal resource allocation, frequently saturating a few collectors leading to gaps in monitored data. Such gaps could prove costly for network operators, and a most common reaction to this is expensive hardware upgrades. In times of such outages, however, it is observed that not all collectors are saturated. More often than not, balancing the load and efficiently distributing available resources was all that was needed to prevent saturation related outages, and increase overall capacity of the overlay. We discuss ways to construct a scalable, robust, and resilient overlay that can function autonomously given the constraints of a typical real world access network.