LightGraph: Lighten Communication in Distributed Graph-Parallel Processing

A number of graph-structured computing abstractions have been proposed to address the needs of solving complex and large-scale graph algorithms. Distributed Graphlab and its successor, PowerGraph, are two such frameworks that have demonstrated excellent performance with high scalability and fault tolerance. However, excessive communication and state sharing among nodes in these frameworks not only reduce network efficiency but may also cause a decrease in runtime performance. In this paper, we first propose a mechanism that identifies and eliminates the avoidable communication during synchronization in existing distributed graph structured computing abstractions. We have implemented our method on PowerGraph and created LightGraph to reduce communication overhead in distributed graph-parallel computation systems. Furthermore, to minimize the required intra-graph synchronizations for PageRank-like applications, LightGraph also employs an edge direction-aware graph partitioning strategy, which optimally isolates the outgoing edges from the incoming edges of a vertex when creating and distributing replicas among different machines. We have conducted extensive experiments using real-world data, and our results verified the effectiveness of LightGraph. For example, when compared with the best existing graph placement method in PowerGraph, LightGraph can not only reduce up to 27.6% of synchronizing communication overhead for intra-graph synchronizations but also cut up to 17.1% runtime for PageRank.