Optimal Stochastic Power Control for Energy Harvesting Systems with Statistical Delay Constraint

This paper studies optimal stochastic power control problem for a time-varying communication link, where the transmitter randomly harvests renewable energies from the environment. The harvested energies are stored in an energy buffer (or battery). Packets arrive at the transmitter data buffer with a constant rate μ. The objective is to maximize μ under the statistical delay and energy harvesting (EH) constraints. In order to study the optimal power control policy, we reformulate the problem as an infinite-horizon Markov decision process (MDP) using asymptotic delay analysis. The optimal policy and its structural properties are studied by employing the post-decision framework approach. We then propose an online power control algorithm, which converges to the optimal solution without requiring the statistical knowledge of the channel fading and EH processes. Numerical results demonstrate the effectiveness of the online algorithm for different delay constraints and EH settings.