Programming Heterogeneous Systems with Array Types

This paper describes the use of array notation called Parray in refinement of parallel programs concerning array type that separates the physical data layout and logical structure of multi-dimensional data, and the control flow diversion of heterogeneous processor units. A case study on matrix multiplication demonstrates refinement of Parray programs: the code evolves from a simple single CPU-thread code to a multi-thread code on CPU/MIC and then a GPU code by modifying the array types in only a few lines of code. A GPU-based SGEMM is implemented in Parray and achieves almost the same Gflops of CUBLAS 4.0 when testing on a single node of Tian-1A system. Because the code operates directly on the logical structure of array, the same SGEMM code can work on different physical array data layouts.