Genetic optimization and hierarchical clustering applied to encrypted traffic identification

An important part of network management requires the accurate identification and classification of network traffic for decisions regarding bandwidth management, quality of service, and security. This work explores the use of a Multi-Objective Genetic Algorithm (MOGA) for both, feature selection and cluster count optimization, for an unsupervised machine learning technique, K-Means, applied to encrypted traffic identification. Specifically, a hierarchical K-Means algorithm is employed, comparing its performance to the MOGA with a non-hierarchical (flat) K-Means algorithm. The latter has already been benchmarked against common unsupervised techniques found in the literature, where results have favored the proposed MOGA. The purpose of this paper is to explore the gains, if any, obtained by increasing cluster purity in the proposed model by means of a second layer of clusters. In this work, SSH is chosen as an example of an encrypted application. However, nothing prevents the proposed model to work with other types of encrypted traffic, such as SSL or Skype. Results show that with the hierarchical MOGA, significant gains are observed in terms of the classification performance of the system.