Scalable evolutionary clustering algorithm with Self Adaptive Genetic Operators

In this paper, we present a scalable evolutionary algorithm for clustering large and dynamic data sets, called Scalable Evolutionary Clustering with Self Adaptive Genetic Operators (Scalable ECSAGO). The proposed evolutionary clustering algorithm can adapt its genetic operators rate while the evolution leads to the optimal centers of the clusters. The sizes of the clusters are estimated using a hybrid analytical optimization procedure. Moreover, a memorization factor is introduced in order to allow the algorithm to keep as much of the previously discovered knowledge about clusters and data summarization as desired. The proposed scalable ECSAGO algorithm is able to find accurate representations of the clusters on very large data sets of different sizes and dimensionality that might not fit in main memory, while maintaining the desirable properties of robustness to noise and automatic detection of the number of clusters. The algorithm is also useful for traking evolving cluster structures that change with the passage of time.