Q-DPM: an efficient model-free dynamic power management technique

When applying dynamic power management (DPM) techniques to pervasively deployed embedded systems, the technique needs to be very efficient so that it is feasible to implement the technique on low end processors and tight-budget memory. Furthermore, it should have the capability to track time varying behavior rapidly, because the time variance is an inherent characteristic of real world systems. Existing methods, which are usually model-based, may not satisfy the aforementioned requirements. In this paper, we propose a model-free DPM technique based on Q-learning. Q-DPM is much more efficient because it removes the overhead of parameter estimator and mode-switch controller. Furthermore, its policy optimization is performed via consecutive online trialing, which also leads to very rapid response to time varying behavior.