A learning theoretic approach to energy harvesting communication system optimization

A machine-to-machine (M2M) system composed of low-power embedded devices powered by energy scavenging mechanisms is considered. The data and energy arrival as well as the channel state processes are all modeled as finite-state Markov processes. Assuming that the state transition probabilities characterizing these processes are unknown at the transmitter, a learning theoretic approach is introduced, and it is shown that the transmitter is able to learn the optimal transmission policy that maximizes the expected sum of the data transmitted during the transmitter´s lifetime. In addition to the learning theoretic approach, online and offline optimization problems are also studied for the same setup. By characterizing the optimal performance for all three problems we identify the loss due to lack of transmitter´s information regarding the behaviors of the underlying processes. Numerical results corroborate theoretical findings and show that, for a given number of learning iterations, the learning theoretic approach reaches a 90% of the performance of the online optimization problem.