On the implementation of broadcast, scatter and gather in a heterogeneous architecture

This paper considers the implementation and evaluation of broadcast, scatter and gather communication operations in heterogeneous machines organized as a Hierarchy of Processor-And-Memory (HPAM). The top levels of the hierarchy consist of a small number of fast processors, whereas the bottom levels consist of a large number of slow processors. Each HPAM level consists of a homogeneous processor mesh. Routing within each level and across levels uses an extended form of X-Y dimension ordered routing. The execution times of three collective communication operations are analytically evaluated in the context of two-level HPAM machines. For each operation, two different alternatives are considered depending on the absence or presence of hardware support. For each alternative, an efficient implementation is characterized. Furthermore, the two alternatives are compared and the gain achieved by using hardware support for these operations is assessed.