Adaptive metric selection for clustering based on consensus affinity

Clustering is one of the most important approaches for organizing huge data in modern times, and many clustering algorithms have been proposed for general or specific tasks. For a certain clustering algorithm, there might be a number of different cases of variance to affect the final clustering quality, in which the selection of metric usually plays the main role. However, it is still an open problem to select a suitable metric for a certain clustering algorithm in an unsupervised manner. To solve this problem, in this paper, we propose a novel method for the task of metric selection for a well-known clustering algorithm, Kmeans. Our method takes advantage of the consensus affinity, which is constructed from a number of individual clustering solutions as those done in cluster ensembles. Notably, compared to traditional cluster ensemble methods, our method avoids solving the cluster ensemble problem, which will result in another selection of related solution methods. Benefiting from the consensus affinity, our proposed method provides significant improvement beyond the average level of investigated metrics. In addition, we conduct the t-test experiments to verify the significance of our proposed method. We also propose to verify the dependence of our methods to related parameters. Studies with experimental validation show the effectiveness and the robustness of our proposed method.