Predictive Blacklisting as an Implicit Recommendation System

A widely used defense practice against malicious traffic on the Internet is through blacklists: lists of prolific attack sources are compiled and shared. The goal of blacklists is to predict and block future attack sources. Existing blacklisting techniques have focused on the most prolific attack sources and, more recently, on collaborative blacklisting. In this paper, we formulate the problem of forecasting attack sources (also referred to as "predictive blacklisting") based on shared attack logs, as an implicit recommendation system. We compare the performance of existing approaches against the upper bound for prediction and we demonstrate that there is much room for improvement. Inspired by the recent NetFlix competition, we propose a multi-level collaborative filtering model that is adjusted and tuned specifically for the attack forecasting problem. Our model captures and combines various factors namely: attacker-victim history (using time-series) and attackers and/or victims interactions (using neighborhood models). We evaluate our combined method on one month of logs from Dshield.org and demonstrate that it improves significantly the prediction rate over state-of-the-art methods as well as the robustness against poisoning attacks.