Bayesian reinforcement learning for energy harvesting communication systems with uncertainty

This paper studies energy harvesting communication systems in which a transmitter sends data packets using the energy harvested from the surrounding natural environment. In many practical situations, the uncertainty of the environment makes the transmitter difficult to keep track of the future change of the physical environment. In addition, the reward value of each successful transmission can be a random variable affected by some factors unknown to the transmitter. We propose a Bayesian reinforcement learning approach for the transmitter to learn the statistic features about the future evolution of the natural environment and probability distribution of the reward value from its previous experience. We derive the optimal policy for the transmitter to sequentially decide its transmit power and the number of transmit data packets to maximize the long-term expected reward. Numerical results show that our proposed algorithm can significantly improve the system performance even when the future environment change and reward value are uncertain for the transmitter.