Title :
A Hierarchical Approach to Maximizing MapReduce Efficiency
Author :
Xiao, Zhiwei ; Chen, Haibo ; Zang, Binyu
Author_Institution :
Parallel Process. Inst., Fudan Univ., Shanghai, China
Abstract :
In this paper, we argued that Hadoop has limitations in exploiting data locality and task parallelism for multi-core platforms. We then extended Hadoop with a hierarchical MapReduce scheme. An in-memory cache scheme is also seamlessly integrated to cache data that is likely to be accessed in memory. Evaluation showed that the hierarchical scheme outperforms Hadoop ranging from 1.4x to 3.5x.
Keywords :
cache storage; multiprocessing systems; parallel processing; Hadoop; MapReduce efficiency; data locality; hierarchical MapReduce scheme; in-memory cache scheme; multicore platform; task parallelism; Distance measurement; Memory management; Multicore processing; Parallel processing; Protocols; Runtime; Servers; MapReduce; hierarchical approach; performance;
Conference_Titel :
Parallel Architectures and Compilation Techniques (PACT), 2011 International Conference on
Conference_Location :
Galveston, TX
Print_ISBN :
978-1-4577-1794-9
DOI :
10.1109/PACT.2011.22
