A round robin with multiple feedback job scheduler in Hadoop

Hadoop is a distributed software platform for processing big data on a large cluster, which implements core mechanism of Google´s GFS and MapReduce. The MapReduce job scheduling algorithm is one of the core technologies of Hadoop. The default job scheduler of Hadoop is FIFO, which will start the job in the order as it is submitted, and this causes the job to be started later when it is submitted later. This paper uses the round robin with a multiple feedback algorithm to solve this problem. With this scheduler, the job which is submitted late, will get quick response and be started without long delay. And the results of experiments on the Hadoop benchmark GridMix indicate that this algorithm can reduce the average response time by 10%-50%.