Reducing Communication and Merging Overheads for Distributed Clustering Algorithms on the Cloud

Many distributed clustering algorithms have been proposed to speed up data clustering on huge database. However, the existing distributed clustering algorithms still suffer from many issues on distributed system such as data synchronization, insufficient scalability, and maintenance difficulties. In this paper, we propose two distributed clustering algorithms named DDC and DGC, which are based on the cloud computing technique. The main ideas of proposed algorithms are to achieve load balance according to an efficient data partition, to cluster more data on many machines in parallel without data dependency, and to merge the result on a machine efficiently with minimal information overlap. The experimental results show that DDC and DGC are able to reduce the execution time and achieve great scalability on the cloud.