Cress: Dynamic Scheduling for Resource Constrained Jobs

Applications on cloud data centers are becoming more complex and various, which makes it necessary to build a flexible and constraint-aware scheduling mechanism. On one hand, dynamic resource allocation according to workloads can achieve higher utilization to the whole system. On the other hand, workload-aware scheduling for special resource can improve the performance of those individual applications. However, it is a challenge to trade off the two goals above, especially for those cloud data centers where an amount of resident services with fluctuating workloads and constraints are running. In this paper, we propose a dynamic scheduling mechanism for resource-constrained jobs. First, a dynamic description language is introduced to describe the multi-dimensional requirements for resource-constrained jobs. Second, a workload-aware resource scheduling algorithm and a conversion method between soft and hard constraints are employed to dynamically adapt to fluctuating workload. Finally, we designed a job scheduler, called Cress, which can dynamically schedule jobs by groups under the hard and soft constraints. The experimental results present that Cress can effectively manage a mixed workload with minimal operation cost and optimal performance.