The Failure-rate Aware Scheduling Policies for Large-scale Cluster Systems

With the scale expanding, node failures become one of the important obstacles when using large-scale cluster systems. The traditional scheduling policies of cluster only took into account the factors such as jobs priority and node load with the node failure rate omitted. The function of job scheduling in cluster system can be divided into two sub-processes: job selection process and node allocation process. In this paper, we introduce several scheduling policies considering the node failure rate with which the more dependable nodes are selected during the node allocation process. In the end, we use the discrete event-driven simulation method to evaluate the policies and the simulation results show that the failure-rate aware scheduling policies do better than random node allocation policy for the system performance