Foreseer: Workload-Aware Data Storage for MapReduce

Author

Jia Zou ; Juwei Shi ; Tongping Liu ; Zhao Cao ; Chen Wang

fYear

2015

fDate

June 29 2015-July 2 2015

Firstpage

746

Lastpage

747

Abstract

Inter-job Write once read many (WORM) scenario is ubiquitous in MapReduce applications that are widely deployed on enterprise production systems. However, traditional MapReduce auto-tuning techniques can not address the inter-job WORM scenario. To address the shortcomings in existing works, this work presents a novel online cross-layer solution, FORESEER. It can automatically predict workloads´ data access information and tune data placement parameters to optimize the over-all performance for an inter-job WORM scenario. In our experiments, we observe that FORESEER can achieve significant performance speedup (up to 37%) compared with previous work.

Keywords

data handling; parallel processing; storage management; Foreseer; MapReduce auto-tuning techniques; data placement parameters; enterprise production systems; inter-job WORM scenario; inter-job write once read many scenario; online cross-layer solution; workload data access information prediction; workload-aware data storage; Clustering algorithms; Distributed databases; Greedy algorithms; Grippers; Optimization; Partitioning algorithms; Writing;

fLanguage

English

Publisher

ieee

Conference_Titel

Distributed Computing Systems (ICDCS), 2015 IEEE 35th International Conference on

Conference_Location

Columbus, OH

ISSN

1063-6927

Type

conf

DOI

10.1109/ICDCS.2015.89

Filename

7164967