Processor allocation policies for reducing resource fragmentation in multi-cluster grid and cloud environments

Multi-cluster is the common underlying architecture of most grid and cloud environments, which usually consist of multiple clusters located at different places. One important characteristic of such computing environments is the performance difference between intra-cluster and inter-cluster communications. Intra-cluster communication networks usually have shorter latency and larger bandwidth than inter-cluster networks. Therefore, in those systems parallel jobs are intended to be executed within a single one of the clusters to achieve better performance although co-allocation across multiple clusters is sometimes technically possible. Under such job execution policy resource fragmentation becomes a crucial issue that happens when there is no single cluster being able to accommodate a job while the total number of processors in the entire grid or cloud is enough for the job. This paper proposes a most-fit policy to reduce resource fragmentation occurrences and evaluates it with several existing processor allocation policies. The experimental results indicate that careful selection of processor allocation policies can improve overall system performance greatly and the proposed most-fit policy can outperform other policies in most conditions.