Dynamic Load Balancing on All-to-All Personalized Communications Using the NNLB Principal

Load balancing is an important factor, especially for non-all-to-all networks of parallel processors (like tori), where there is no direct communication between all nodes, thus contentions for certain links may arise. Load balancing can assure of shortest waiting times for channel distribution, thus shortest latencies. Also it optimizes resource use, maximizes throughput, and facilitates the pipelining of messages, a technique that reduces the transfer times. This paper presents a communication scheme based on modular arithmetic. The model applies the Nearest Neighbor Load Balancing (NNLB) principle to achieve dynamic load balancing on non-all-to-all networks, when the communication pattern adopted is all-to-all personalized communication. Results show that redistributing equal-sized messages in every step produces better results in terms of communication cost.