An Adaptive Resource Sharing Platform for Internet Data Center

Internet applications are diverse, such as online web services, offline data analysis jobs and so on. To make different types of applications work effectively, many programming models and computing frameworks are designed, such as MapReduce, Dryad, and Hadoop. How to support the emerging applications and computing frameworks with high resource utilization is a serious challenge. In this paper, we present a resource sharing platform which can support different types of applications and computing frameworks by adopting a job-resource description protocol. To improve the resource utilization without impairing the performances of applications, we propose a job-resource matching algorithm, called JRMatching, which deploys those different types of jobs on one node hybridly to reduce resource fragments, and a mechanism to provide resources adaptively, which can reduce resource waste caused by fixed resource allocation. The experiments show that the performances of applications sharing resources in one node are close to running exclusively and the resource utilization of the internet data center also can be improved.