DocumentCode :
1805755
Title :
ActCap: Accelerating MapReduce on heterogeneous clusters with capability-aware data placement
Author :
Bo Wang ; Jinlei Jiang ; Guangwen Yang
Author_Institution :
Dept. of Comput. Sci. & Technol., Tsinghua Univ., Beijing, China
fYear :
2015
fDate :
April 26 2015-May 1 2015
Firstpage :
1328
Lastpage :
1336
Abstract :
As a widely used programming model and implementation for processing large data sets, MapReduce performs poorly on heterogeneous clusters, which, unfortunately, are common in current computing environments. To deal with the problem, this paper: 1) analyzes the causes of performance degradation and identifies the key one as the large volume of inter-node data transfer resulted from even data distribution among nodes of different computing capabilities, and 2) proposes ActCap, a solution that uses a Markov chain based model to do node-capability-aware data placement for the continuously incoming data. ActCap has been incorporated into Hadoop and evaluated on a 24-node heterogeneous cluster by 13 benchmarks. The experimental results show that ActCap can reduce the percentage of inter-node data transfer from 32.9% to 7.7% and gain an average speedup of 49.8% when compared with Hadoop, and achieve an average speedup of 9.8% when compared with Tarazu, the latest related work.
Keywords :
Markov processes; data handling; parallel programming; ActCap; MapReduce acceleration; Markov chain; Tarazu; data distribution; heterogeneous clusters; inter-node data transfer; node-capability-aware data placement; Benchmark testing; Computational modeling; Computers; Conferences; Data transfer; Hardware; Markov processes; Big Data; Data Placement; Heterogeneous Clusters; Load Balancing; MapReduce;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Computer Communications (INFOCOM), 2015 IEEE Conference on
Conference_Location :
Kowloon
Type :
conf
DOI :
10.1109/INFOCOM.2015.7218509
Filename :
7218509
Link To Document :
بازگشت