The self-tuning dynP job-scheduler

In modern resource management systems for supercomputers and HPC-clusters the job-scheduler plays a major role in improving the performance and usability of the system. The performance of the used scheduling policies (e.g. FCFS, SJF, LJF) depends on the characteristics of the queued jobs. Hence we developed the dynP scheduler family. The basic idea was to change between different scheduling policies during runtime. The basic dynP scheduler uses the average estimated runtime of all queued jobs together with two input parameters to decide when a policy change may be benefical. A disadvantage is that the performance of the basic dynP scheduler strongly depends on the right setting of the two input parameters. Therefore we present the self-tuning dynP scheduler, which is totally independent from any parameter values. The basic concept is that the self-tuning dynP scheduler computes the full (virtual) schedule for each of the three policies in every scheduling step. Each computed schedule is rated by a criterion. Then the scheduler switches to that policy which generated the best schedule for the currently queued jobs. In this paper we are using simulations with trace based job sets to evaluate the performance of the scheduler. The achieved results are reasonably good compared to the parameterized dynP variant and the basic policies FCFS, SJF, and LJF