A (Radical) Proposal Addressing the Non-scalability of the Irregular MPI Collective Interfaces

In both the regular and the irregular MPI collectives there is a correspondence between the argument lists and certain MPI derived datatypes. As a means to address and alleviate well-known memory and performance scalability problems in the irregular (or vector) collective interface definitions of MPI we propose to push this correspondence to its natural limit, and replace the interfaces of the MPI collectives with a new set of interfaces that specify all data sizes and displacements solely by means of derived datatypes. This reduces the number of collective (communication and reduction) interfaces from 16 to 10 (in the extreme to only 7), significantly generalizes the operations, unifies regular and irregular collective interfaces, makes it possible to decouple certain algorithmic decisions from the collective operation, and moves the interface scalability issue from the collective interfaces to the MPI derived datatypes. To complete the proposal we discuss the memory scalability of the derived datatypes and suggest a number of new datatypes for MPI, some of which may be of independent interest. Finally, we introduce additional, new functionality for scalable construction of datatypes as needed for the new collective interfaces. We give examples throughout of the benefits of this alternative set of collective interfaces. It is argued that an implementation can straightforwardly be undertaken within any reasonable MPI implementation.