A heuristic fault tolerant MapReduce framework for minimizing makespan in Hybrid Cloud Environment

Cloud Computing propounds a striking option for business to pay only for the resources that were consumed. The prime challenge is to increase the MapReduce clusters to minimize their costs. MapReduce is a widely used parallel computing framework for large scale data processing. The major concern of map reduce programming model are job execution time and cluster throughput. Multiple speculative execution strategies have been proposed, but all are failed to address the DAG communication and cluster utilization. In this paper, we developed a new strategy, OTA (Optimal Time Algorithm), which improves the effectiveness of speculative execution significantly. OTA do not consider the difference between the execution time of tasks on the same processors, they may form clusters of tasks that are not similar to each other. The proposed strategy efficiently utilizes the characteristics and properties of the MapReduce jobs in the given workload for constructing optimal job schedule. This resolves the problem of minimizing the makespan of workloads that additionally includes the workflow (DAGs) of mapreduce jobs.