GPU-centered parallel model on heterogeneous multi-GPU clusters

On the multi-GPU cluster platform, it is difficult to use the full compute power of the CPUs. One of the reasons is that the traditional parallel models based on the homogeneous platform is not suitable to the heterogeneous platform. We research and develop the GPU-centered parallel model to control the CPUs using more fine granularity. This model decreases the idle time of the CPUs introduced by the work load unbalance significantly. Our experiments show that this model can achieve higher performance than the traditional node-centered parallel model for some real applications. The efficiency of the LINPACK benchmark using GPU-centered parallel model is 5.34% higher than the node-centered parallel model.