Performance impact of job arrivals on clusters and grids through realistic model-based simulation

Performance evaluation of schedulers in large-scale parallel and distributed systems such as clusters and grids is an essential and crucial issue because job scheduling is a fundamental operation on these systems. Since workloads are an indispensable part during the evaluation process, they are considered as one of the key factors that can affect the reliability of the results. Statistical studies have shown that real cluster and grid workloads exhibit several important and correlated characteristics. Using workloads with/without the observed characteristics in an evaluation can lead to incorrect conclusions since they may potentially have severe impacts on scheduling performance. In this paper, we study how long range dependence (LRD) and burstiness structures of job arrivals in a workload affect the performance of clusters and grids through realistic model-based simulations. Representative statistical workload models able to capture these characteristics are used to generate realistic workloads, which are then run by means of simulation to obtain dependable results. Our experiments show that background workloads with dependence and burstiness cause a performance degradation of a cluster but seem to have little impact on a grid. In contrast, grid workloads with these properties do not affect the scheduling performance of a cluster but decrease that of a grid. Furthermore, we also indicate that using workloads with/without LRD and burstiness in evaluating cluster schedulers may produce different results.