Achieving Context Awareness and Intelligence in Distributed Cognitive Radio Networks: A Payoff Propagation Approach

Cognitive Radio (CR) is a next-generation wireless communication system that exploits underutilized licensed spectrum to optimize the utilization of the overall radio spectrum. A Distributed Cognitive Radio Network (DCRN) is a distributed wireless network established by a number of CR hosts in the absence of fixed network infrastructure. Context awareness and intelligence are key characteristics of CR networks that enable the CR hosts to be aware of their operating environment in order to make an optimal joint action. This research aims to achieve context awareness and intelligence in DCRN using our novel Locally-Confined Payoff Propagation (LCPP), which is an important feature in Multi-Agent Reinforcement Learning (MARL). The LCPP mechanism is suitable to be applied in most applications in DCRN that require context awareness and intelligence such as scheduling, congestion control, as well as Dynamic Channel Selection (DCS), which is the focus of this paper. Simulation results show that the LCPP mechanism is a promising approach. The LCPP mechanism converges to an optimal joint action including networks with cyclic topology. Fast convergence is possible. The investigation in this paper serve as an important foundation for future work in this research field.