Predictive Data and Energy Management under Budget

Power reducing in clusters has become increasingly important over the past few years. People have tried hard to reduce the power consumption of clusters. However, managing the power is more important than reducing the power. In this paper, we add power consumption to the list of managed resources and help developers to understand and control power profile of their clusters. MapReduce is an efficient and popular programming model for data-intensive computing, so we focus on designing green power management for MapReduce workloads. We designed these strategies to make every node in clusters run under a local power budget, and the whole cluster under a global power budget. We modified the data placement policies in HDFS, designed dynamic replica placement policies, and examined different workloads to learn power consumption models. In addition, we also right sizing the clusters according to the power budget. As our predictive power model focuses on the variation of the power, we can predict when users should take measures to reduce power usage. We also present implementation and experiments in this paper.