Modeling I/O Interference in Data Intensive Map-Reduce Applications

Author

Groot, Sven

Author_Institution

Inst. of Ind. Sci., Univ. of Tokyo, Tokyo, Japan

fYear

2012

fDate

16-20 July 2012

Firstpage

206

Lastpage

209

Abstract

Map-Reduce is a popular framework for very large-scale data mining and processing. Recently, some works have attempted to model the behavior of Map-Reduce, but these existing models ignore the non-linearity of disk I/O performance under contention, which is a critical aspect of estimating the performance of data intensive applications. Understanding I/O interference between tasks running on the same node is critical in optimizing task scheduling for improved resource utilization. In this paper, we present a model to estimate the I/O behavior of Map-Reduce applications that can be used to achieve these goals.

Keywords

cloud computing; data mining; resource allocation; scheduling; I/O behavior estimation; I/O interference modeling; cloud computing; data intensive MapReduce application; disk I/O performance nonlinearity; resource utilization; task scheduling optimization; very large-scale data mining; very large-scale data processing; Cloud computing; Computational modeling; Data models; Heart beat; Interference; Predictive models; Resource management; Map-Reduce; cloud computing; data intensive;

fLanguage

English

Publisher

ieee

Conference_Titel

Applications and the Internet (SAINT), 2012 IEEE/IPSJ 12th International Symposium on

Conference_Location

Izmir

Print_ISBN

978-1-4673-2001-6

Electronic_ISBN

978-0-7695-4737-4

Type

conf

DOI

10.1109/SAINT.2012.88

Filename

6305283