Title :
k-Balanced sorting and skew join in MPI and MapReduce
Author :
Silu Huang ; Fu, Ada Wai-Chee
Author_Institution :
Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Hong Kong, China
Abstract :
We consider algorithms for sorting and skew equi-join operations for computer clusters. The proposed algorithms achieve the best known theoretical workload balancing guarantee, and exhibit close to optimal balancing in our experiments. Our empirical studies also show that the proposed sorting algorithm is up to 30% faster than the state-of-the-art algorithm.
Keywords :
application program interfaces; message passing; resource allocation; sorting; workstation clusters; MPI; MapReduce; computer clusters; k-balanced sorting algorithm; skew equi-join operations; skew join; workload balancing; Clustering algorithms; Computers; Parallel algorithms; Partitioning algorithms; Runtime; Silicon; Sorting;
Conference_Titel :
Big Data (Big Data), 2014 IEEE International Conference on
Conference_Location :
Washington, DC
DOI :
10.1109/BigData.2014.7004237