Efficient Parallel Community Detection in Large Edge-Intensive Networks

Community detection is a classic and very difficult task in social network analysis. A large number of methods have been developed for both efficient and effective community detection. However, much of the existing methods are heavily dependent on the number of links in the network, and thus they often suffer from the computational inefficiency when meeting large edge-intensive networks. In this paper, we present a novel SIMPLifying and Ensembling (SIMPLE) framework for parallel community detection. It employs the random link sampling to simplify the network and obtain basic partitionings on every sampled graphs. Then, the K-means-based Consensus Clustering is used to ensemble a number of basic partitionings to get high-quality community structures. Meanwhile, steps of random sampling and sampled graph partitioning are encapsulated into MapReduce to further improve the efficiency. Experiments on four real-world social networks analyze key parameters and factors inside SIMPLE, and demonstrate the effectiveness of the SIMPLE.