Title :
Evaluating task scheduling in hadoop-based cloud systems
Author :
Shengyuan Liu ; Jungang Xu ; Zongzhen Liu ; Xu Liu
Author_Institution :
Coll. of Comput. & Control Eng., Univ. of Chinese Acad. of Sci., Beijing, China
Abstract :
Nowadays, private clouds are widely used for resource sharing. Hadoop-based clusters are the most popular implementations for private clouds. However, because workload traces are not publicly available, few previous work compares and evaluates different cloud solutions with publicly available benchmarks. In this paper, we use a recently-released Cloud benchmarks suite - CloudRank-D to quantitatively evaluate five different Hadoop task schedulers, including FIFO, capacity, naïve fair sharing, fair sharing with delay, and HOD (Hadoop On Demand) scheduling. Our experiments show that with an appropriate scheduler, the throughput of a private cloud can be improved by 20%.
Keywords :
cloud computing; resource allocation; scheduling; CloudRank-D; FIFO; HOD; Hadoop On Demand scheduling; Hadoop task schedulers; Hadoop-based cloud systems; Hadoop-based clusters; capacity; naive fair sharing; private clouds; released Cloud benchmarks suite; resource sharing; task scheduling evaluation; workload traces; Benchmark testing; Cloud computing; Data mining; Data warehouses; Delays; Scheduling; Throughput; Cloud; Evaluation; Hadoop; Taks scheduling;
Conference_Titel :
Big Data, 2013 IEEE International Conference on
Conference_Location :
Silicon Valley, CA
DOI :
10.1109/BigData.2013.6691697
