Detect and analyze Large-scale BGP events by bi-clustering Update Visibility Matrix

Many attempts have been made to detect and analyze anomalous Internet events through dissecting BGP updates and tables, and substantial progress has been made in detecting and quantifying the impact of major Internet disruptions. However, we notice that most works in this realm either deploy/use a limited quantity of monitors or analyze aggregated statistics, and such practice may result in overestimating the impact of monitor-local events, which can be viewed only by a rather small portion of the Internet. To eliminate the impact of such local events on the detection of Internet-level anomalies, we raise the concept of Large-scale BGP Event (LBE), which affects a large amount of IP prefixes (high impact) and is widely observable (non-local). To detect LBE, we record update data in the Update Visibility Matrix (UVM) according to the prefix and monitor related to each update. At first, we formulate the problem of identifying LBE in UVM as a bi-clustering problem; after proving it is NP-hard, we describe our heuristic algorithm. Next, we apply our scheme to more than 2 TB of historical data. We find that LBE is highly correlated with many well-known disruptive incidents. Furthermore, we also identify some abnormal events that have never been investigated. We believe our work can assist in network operation tasks such as problem prevention, diagnosis, and recovery.