MIMO Transmission Control in Fading Channels—A Constrained Markov Decision Process Formulation With Monotone Randomized Policies

This paper addresses the optimal power and rate allocation control in multiple-input multiple-output (MIMO) wireless systems over Markovian fading channels. The problem is posed as an infinite horizon average-cost constrained Markov decision process (CMDP) with the goal of minimizing the average transmission power subject to delay constraints. By using a Lagrangian formulation of the CMDP, we use the concepts of stochastic dominance, submodularity, and multimodularity to prove that the optimal randomized policies are monotone. Three important structural results on the nature of the optimal randomized policies are derived. First, we show that the action space can be exponentially reduced by decomposing the rate allocation problem into bit-loading problem across individual antennas and the total rate allocation based on the current buffer occupancy and channel state. Second, we show that the optimal rate allocation policy is a randomized mixture of two pure policies that are monotonically increasing in the buffer occupancy. Finally, we show that the optimal power allocation is piecewise linear in the delay constraint. These three structural results can be exploited to devise efficient online reinforcement learning algorithms for optimal rate allocation.