DocumentCode :
2876393
Title :
A simulation approach to evaluating design decisions in MapReduce setups
Author :
Wang, Guanying ; Butt, Ali R. ; Pandey, Prashant ; Gupta, Karan
Author_Institution :
Virginia Tech, Blacksburg, VA, USA
fYear :
2009
fDate :
21-23 Sept. 2009
Firstpage :
1
Lastpage :
11
Abstract :
MapReduce has emerged as a model of choice for supporting modern data-intensive applications. The model is easy-to-use and promising in reducing time-to-solution. It is also a key enabler for cloud computing, which provides transparent and flexible access to a large number of compute, storage and networking resources. Setting up and operating a large MapReduce cluster entails careful evaluation of various design choices and run-time parameters to achieve high efficiency. However, this design space has not been explored in detail. In this paper, we adopt a simulation approach to systematically understanding the performance of MapReduce setups. The resulting simulator, MRPerf, captures such aspects of these setups as node, rack and network configurations, disk parameters and performance, data layout and application I/O characteristics, among others, and uses this information to predict expected application performance. Specifically, we use MRPerf to explore the effect of several component inter-connect topologies, data locality, and software and hardware failures on overall application performance. MRPerf allows us to quantify the effect of these factors, and thus can serve as a tool for optimizing existing MapReduce setups as well as designing new ones.
Keywords :
Internet; digital simulation; pattern clustering; scheduling; software engineering; MRPerf simulator; MapReduce setups; cloud computing; component inter-connect topologies; data locality; design decisions evaluation; hardware failures; modern data-intensive applications; simulation approach; software failures; time-to-solution reduction; Application software; Cloud computing; Computational modeling; Computer networks; Design optimization; Hardware; Network topology; Predictive models; Runtime; Software performance;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Modeling, Analysis & Simulation of Computer and Telecommunication Systems, 2009. MASCOTS '09. IEEE International Symposium on
Conference_Location :
London
ISSN :
1526-7539
Print_ISBN :
978-1-4244-4927-9
Electronic_ISBN :
1526-7539
Type :
conf
DOI :
10.1109/MASCOT.2009.5366973
Filename :
5366973
Link To Document :
بازگشت