Scheduling diverse high performance computing systems with the goal of maximizing utilization

High performance computing resources attract a wide range of computational users and corresponding job widths and lengths. For example, on the petaflop Cray XT5 machine, Kraken, users submit jobs ranging from a few hundred cores (capacity computing) to over hundred thousand cores (capability computing). Traditionally it has been difficult to maintain high utilization while juggling such a diverse job mix. This paper explores four unique approaches to achieve our scheduling goals of maximizing utilization on four distinct resources at the National Institute for Computational Sciences. The resources include the petaflop machine, Kraken, Athena - a 166 TF Cray XT4, a 4 TB shared memory NUMA machine called Nautilus, and a GPU cluster called Keeneland.