Big Data Analytics for Detecting Host Misbehavior in Large Logs

The management of complex network infrastructures continues to be a difficult endeavor today. These infrastructures can contain a huge number of devices that may misbehave in unpredictable ways. Many of these devices keep logs that contain valuable information about the infrastructures´ security, reliability, and performance. However, extracting information from that data is far from trivial. The paper presents a novel approach to assess the security of such an infrastructure using its logs, inspired on data from a real telecommunications network. We use machine learning and data mining techniques to analyze the data and semi-automatically discover misbehaving hosts, without having to instruct the system about how hosts misbehave.