Load balancing hybrid programming models for SMP clusters and fully permutable loops

This paper emphasizes on load balancing issues associated with hybrid programming models for the parallelization of fully permutable nested loops onto SMP clusters. Hybrid parallel programming models usually suffer from intrinsic load imbalance between threads, mainly because most existing message passing libraries generally provide limited multi-threading support, allowing only the master thread to perform internode message passing communication. In order to mitigate this effect, the authors proposed a generic method for the application of static load balancing on the coarse-grain hybrid model for the appropriate distribution of the computational load to the working threads. The efficiency of the proposed scheme was experimentally evaluated against a micro-kernel benchmark, and demonstrated the potential of such load balancing schemes for the extraction of maximum performance out of hybrid parallel programs.