Towards an Efficient Multi-way Factorization of Multi-dimensional Big Data across a GPU Cluster

It has long been an important issue in various disciplines to examine massive multi-dimensional data by extracting the embedded multi-way factors. With the quick increases in both scales and dimensions of data under analysis, research challenges arise in order to reflect the dynamics of large-scale tensors while introducing no significant distortions in the factorization procedure in sophisticated applications. A massively parallel computing framework, namely H-PARAFAC, has been developed to enable Parallel Factor Analysis (PARAFAC) of massive tensors upon a "divide-and-conquer" theory (a modified alternating least squares approach). The hierarchical framework incorporates a coarse-grained model for coordinating the processing of sub tensors and a fine-grained parallel model for computing each sub tensor and fusing sub-factors. Experiments have been performed on a GPU cluster, and the results indicate that (1) the proposed method breaks the limitation on the size of data to be factorized, and (2) it dramatically outperforms the traditional counterparts in terms of both scalability and efficiency, e.g., The runtime increases linearly with the data volume increases in the order of n3.