Dynamic Deployment and Management of Elastic Virtual Clusters

Virtual clusters are the fundamental support of new generation virtual High Performance Computing (HPC) systems. As a high level virtual system component, virtual cluster is designed based on virtual machine and virtual network to provide virtual executing environment for large-scale parallel applications. This paper proposes a novel solution for dynamic deployment and customization of Elastic Virtual Cluster (EVC). In this work, we support customization of virtual clusters, such as OS images, network topologies and cluster software. Virtual clusters are automatically configured with isolated virtual networks (network address, DNS domain, NIS domain, etc.) and software environments. Two data transmission protocols are presented in this paper to accelerate image deployment. In addition, we propose a novel solution that virtual machines (VMs) on the same host share image without the aid of VMM. Experiment results show that virtual clusters can be deployed and managed on distributed physical nodes efficiently and therefore can be used by the upper level resource consumers or applications.