A Compiler Transformation to Overlap Communication with Dependent Computation

Hiding communication latency is essential to achieve scalable performance on current and future parallel systems. In this extended abstract, we present a novel compiler transformation that overlaps communication with computation to hide communication latency. Unlike prior work, we are able to achieve this overlap even in the presence of an overlap-inhibiting data dependence between the communication and computation. We do so by transforming the data dependence into an overlap-amenable one. To achieve this overlap, the Maunam compiler transforms the code by employing array expansion, partial loop peeling, loop alignment, and array contraction. This transformation is useful for optimization of systolic, communication avoiding algorithms.