Learning through Reinforcement for Repeated Power Control Game in Cognitive Radio Networks

This paper studies the repeated power control game in cognitive radio (CR) networks through reinforcement learning without channel and power strategy information exchange among CR users. Unlike traditional game-theoretical approaches on CR power control, this research solves the incomplete information power control problems for selfish and autonomous CR users for the first time. Each CR user in the problem only knows its own channel and power strategy while the information of primary users (PUs) and other different types of CR users are unknown. The formulated power control problem is a constrained repeated stochastic game with learning automaton. The objective of this repeated game is to maximize the average utility of each CR user under the interference power constraints of PUs. At each time step, the CR user only knows its own utility and the interference functions after the play but no further information. This power control game is proved to be asymptotically equivalent to the traditional game theory approaches. The properties of existence, diagonal concavity and uniqueness for this game are illustrated in detail. A Bush-Mosteller reinforcement learning procedure is designed for the power control algorithm. Finally, the learning based power control algorithm is implemented, and the simulation results with detailed analysis are shown to enforce the effectiveness of the proposed algorithms.