DocumentCode :
249325
Title :
Marimba: A Framework for Making MapReduce Jobs Incremental
Author :
Schildgen, Johannes ; Jorg, Thomas ; Hoffmann, Marco ; Dessloch, Stefan
Author_Institution :
Univ. of Kaiserslautern, Kaiserslautern, Germany
fYear :
2014
fDate :
June 27 2014-July 2 2014
Firstpage :
128
Lastpage :
135
Abstract :
Many MapReduce jobs for analyzing Big Data require many hours and have to be repeated again and again because the base data changes continuously. In this paper we propose Marimba, a framework for making MapReduce jobs incremental. Thus, a recomputation of a job only needs to process the changes since the last computation. This accelerates the execution and enables more frequent recomputations, which leads to results which are more up-to-date. Our approach is based on concepts that are popular in the area of materialized views in relational database systems where a view can be updated only by aggregating changes in base data upon the previous result.
Keywords :
Big Data; parallel programming; relational databases; Big Data analysis; Marimba framework; base data; change aggregation; incremental MapReduce jobs; job recomputation; materialized views; relational database systems; Aggregates; Big data; Computational modeling; Google; Programming; Relational databases; Rhythm; Hadoop; MapReduce; framework; incremental;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Big Data (BigData Congress), 2014 IEEE International Congress on
Conference_Location :
Anchorage, AK
Print_ISBN :
978-1-4799-5056-0
Type :
conf
DOI :
10.1109/BigData.Congress.2014.27
Filename :
6906770
Link To Document :
بازگشت