Robustness of Community Partition Similarity Metrics

Currently, detecting communities in real-world networks is a problem of considerable interest and many community detection algorithms have been proposed. In this paper, we study a number of community partition similarity measures to measure the the robustness and performances of different community detection algorithms. By carrying out a careful comparative analysis of 3 common used community partition similarity measures, to our surprise, these metrics all have systematical biases. To get more details of the widely used partition similarity measures, we show that the bias partitions of these 3 different widely used partition similarity measures, e. g., normalized mutual information, Jaccard index and normalized van Dongen metric in some extreme invalid cases. Finally, we propose a new similarity metric to evaluate the accuracy of community partitions. Our metric performs well for testing the accuracy and robustness of community detection algorithms in all cases.