Programming GPU Clusters with Shared Memory Abstraction in Software

As many-core graphics processors gain an increasingly important position concerning the advancements on modern highly concurrent processors, we are experiencing the deployment of the first heterogeneous clusters that are based on GPUs. The attempts to match future expectations in computational power and energy saving with hybrid - GPU-based - clusters are expected to grow in the next years, and much of their success will depend on the provision of the appropriate programming tools. In the current paper we propose a programming model for GPU clusters that is based on shared memory abstraction. We give evidence for the applicability of the proposed model under two cases. In the first case we describe an implementation procedure that involves the utilization of Intel Cluster OpenMP, a cluster-enabled OpenMP implementation. Subsequently, we present an extended version of Pleiad, a cluster middleware which is based on the Java platform. The evaluation of these schemes under two characteristic computationally intensive applications on a 4-node multi-GPU cluster, reveals that such approaches can easily enhance existing GPU software development tools, such as CUDA, and they can lead to a significant acceleration of applications that can benefit from many-core GPU clusters.