Adaptive scheduling in the cloud — SLA for Hadoop job scheduling

Hadoop is a progressively indispensable cloud-computing platform that several vendors have been offering as a service. When a consumer submits a job to Hadoop, there is no guarantee that the job will finish in a required amount of time. Given the immediate need to develop a mechanism to serve the Hadoop service with a standardized agreement between vendor and consumer, this paper proposes the Adaptive Scheduler (AS). The AS requires consumers to submit a Service Level Agreement (SLA) together with a job. The SLA is used to check whether the vendor can accommodate the job to meet the SLA. If it can, then the AS schedules and executes the job using the SLA. If not, the consumer is asked to negotiate with the AS to come up with an SLA that both parties could agree upon; pre-agreements between vendors and consumers benefit both parties. A comparative study of several existing schedulers in Hadoop demonstrates the advantages of the proposed AS.