Performance Model for Parallel Matrix Multiplication with Dryad: Dataflow Graph Runtime

In order to meet the big data challenge of today´s society, several parallel execution models on distributed memory architectures have been proposed: MapReduce, Iterative MapReduce, graph processing, and dataflow graph processing. Dryad is a distributed data-parallel execution engine that model program as dataflow graphs. In this paper, we evaluated the runtime and communication overhead of Dryad in realistic settings. We proposed a performance model for Dryad implementation of parallel matrix multiplication (PMM) and extend the model to MPI implementations. We conducted experimental analyses in order to verify the correctness of our analytic model on a Windows cluster with up to 400 cores, Azure with up to 100 instances, and Linux cluster with up to 100 nodes. The final results show that our analytic model produces accurate predictions within 5% of the measured results. We proved some cases that using average communication overhead to model performance of parallel matrix multiplication jobs on common HPC clusters is the practical approach.