Scheduling Data Parallel Workloads - A Comparative Study of Two Common Algorithmic Approaches

The dynamic loop scheduling (DLS) and the divisible load theory (DLT) are two common algorithmic approaches used in the scheduling of arbitrarily divisible workloads. Despite sharing the same goal of achieving load balancing via scheduling, they are fundamentally different. Specifically, the DLS approach is probabilistic and platform agnostic, whereas the DLT approach is deterministic and platform aware. To the best of our knowledge, this is the first work to conduct a comparative study and a performance analysis of the two approaches. The study is beneficial for identifying the application, algorithmic, and systemic characteristics that favor one approach over the other. In this work, we report the results of a comparative performance study of the two approaches. Simulations provide a greater flexibility and control over running experiments on a real computing platform. Hence, we employ simulations in this work to study the behavior of the DLS and the DLT approaches on two types of network topologies, namely a single level tree network and a linear array network. Various application, algorithmic, and systemic characteristics introduce load imbalance, and therefore we use a wide range of synthetically generated workloads and variable system conditions to inject load imbalance into the simulated platform. The simulation results demonstrate the effectiveness of applying the DLS approach, when the computing environments are mainly characterized by high load imbalance, and that of applying the DLT approach when they are mainly characterized by high communication costs.