Automatic VM Allocation for Scientific Application

Cloud has been the main technology utilized as a high performance computing (HPC) platform. The characteristics of cloud can satisfy a large scale processing required by scientific applications, which are mostly compute-intensive with big data. Cloud can also reduce the computing cost through sharing and virtualizing of resources. In the cloud, a large number of virtual machines (VM) can be generated on demands. In order to obtain the optimal cost and high efficiency in the task execution on the public cloud, the suitable amount of virtual machines should be properly determined prior to the start of the computation. Moreover, the application should be effectively partitioned and distributed onto the virtual machines. In this paper, we propose an automatic mechanism to allocate the optimal numbers of resources in the cloud. The novel resource estimation model and scheduling algorithm are presented. We select an analytic application with high level of computations in the field of epidemic forecast to demonstrate the use of the designed mechanism. Experimental studies have been conducted to examine the resource prediction accuracy and the scalability of running the application on the cloud.