Scalable Loop Self-Scheduling Schemes Implemented on Large-Scale Clusters

Loops are the largest source of parallelism in many scientific applications. Parallelization of irregular loop applications is a challenging problem to achieve scalable performance on large-scale multi-core clusters. Previous research proposed an effective Master-Worker model on clusters for distributed self scheduling schemes that apply to parallel loops with independent iterations. However, this model has not been applied to large-scale clusters. In this paper, we present an extension of the distributed self-scheduling schemes implemented in a hierarchical Master-Worker model. Our experiments with different self-scheduling schemes demonstrate good scalability when scaling up to 8, 192processors.