Multiround algorithms for scheduling divisible loads

Divisible load applications occur in many fields of science and engineering and can be easily parallelized in a master-worker fashion, but pose several scheduling challenges. While a number of approaches have been proposed that allocate load to workers in a single round, using multiple rounds improves overlap of computation with communication. Unfortunately, multiround algorithms are difficult to analyze and have thus received only limited attention. In this paper, we answer three open questions in the multiround divisible load scheduling area: 1) how to account for latencies, 2) how to account for heterogeneous platforms, and 3) how many rounds should be used. To answer 1), we derive the first closed-form optimal schedule for a homogeneous platform with both computation and communication latencies, for a given number of rounds. To answer 2) and 3), we present a novel algorithm, UMR. We evaluate UMR in a variety of realistic scenarios.