Development of a Disk-Power Monitoring Environment Using a Stream Processing Engine

Rapid increase of digital information accelerates advancement of large capacity and high performance storage systems. Reduction of power-consumption in storage systems is also paid much attention. For improvement of performance and power consumption, storage management techniques utilizing sensing data provided by power meters and operating systems become important. However, we have to collect and process massive sensing data from heterogeneous sources in storage systems. In this paper, to reduce costs for implementing storage management techniques and their evaluation, we propose a disk monitoring environment using a stream processing engine. Stream processing engines are specialized to process continuous queries on data streams. We have developed some wrapper programs to import sensing data. Our monitoring environment covers power-consumption of each disk, disk temperature, file accesses on a file system, IO statistics and so on. Users can specify monitoring requirements for these streams as continuous queries. We also consider changes of hardware configurations such as the number of disks to be monitored. We can dynamically change parameters for wrappers, and add/remove wrappers on the running stream processing engine.