Resource selection and allocation for dynamic adaptive computing in heterogeneous clusters

This paper provides a framework for dynamic adaptive computing in heterogeneous clusters for computationally intensive applications. The framework considers a set of discoverable interconnected computational resources and either a parallel or sequential workload needing to be executed. An adaptive inclusion/exclusion algorithm is used to select the resources by using novel performance measurements and profiling techniques. Furthermore, contrary to a greedy approach where all the resources are seized for the workload application, our framework only harnesses the best fit resources measured against system-wide performance characterization, and is contingent upon the current workload definition. The intelligent selection of a subset of resources has proven to achieve better performance; especially in environments with a high level of heterogeneity where the characteristics of some resources may not achieve the best performance the cluster can provide. Additionally, this paper provides a novel analysis of the workload and cluster characteristics, exhibiting analytical starting points to be used in the resource selection.