Exploiting Resource Usage Patterns for Better Utilization Prediction

Understanding the resource utilization in computing clouds is critical for efficient resource planning and better operational performance. In this paper, we propose two ways, from microscopic and macroscopic perspectives, to predict the resource consumption for data centers by statistically characterizing resource usage patterns. The first approach focuses on the usage prediction for a specific node. Compared to the basic method of calibrating AR models for CPU usages separately, we find that using both CPU and memory usage data can improve the forecasting performance. The second approach is based on Principal Component Analysis (PCA) to identify resource usage patterns across different nodes. Using the identified patterns, we can reduce the number of parameters for predicting the resource usage on multiple nodes. In addition, using the principal components obtained from PCA, we propose an optimization framework to optimally consolidate VMs into a number of physical servers and in the meanwhile reduce the resource usage variability. The evaluation of the proposed approaches is based on traces collected from a production cloud environment.