Kernel-Assisted MPI Collective Communication among Many-core Clusters

Architectural hierarchies and hardware complexity brought by multicore or many-core Clusters, greatly challenge MPI applications´ performance in two ways: performance efficiency and cross-platform portability. The cross-platform portability assumption, ´write once and efficiently run everywhere,´ is not guaranteed by current MPI libraries, mainly due to implementation details. To partially address the performance issue exposed by hardware complexity and memory hierarchies, we propose a kernel assisted MPI collective communication approach, directly based on the kernel assisted one-sided single-copy module: KNEM. First, we introduce the general operating principles of KNEM memory copy, and then we present the design and implementation of KNEM collective, an intra-node collective component of Open MPI for shared memory nodes. Additionally, we describe how to integrate the kernel-assisted approach into collective communications on heterogeneous multicore clusters with intra- and inter-node communication. We evaluate and experimentally demonstrate the performance advantages of our kernel-assisted MPI collective over state-of-art MPI libraries (Open MPI and MVAPICH2).