Route selection for minimizing interference to primary users in Cognitive Radio Networks: A Reinforcement Learning approach

The dynamicity and unpredictability of channel availability in Cognitive Radio Networks (CRNs) have imposed a major challenge to the design of routing schemes. Generally speaking, routing enables Secondary Users (SUs) to select optimal routes, while minimizing interference to Primary Users (PUs), which is a key requirement of CRNs. Nevertheless, there has been limited research in the literature for routing schemes that consider both SUs´ interference to PUs and network-wide performance of SUs. This paper proposes a novel Cognitive Radio Q-routing (CRQ-routing) scheme, which aims to take account of SU-to-PU interference and SUs´ end-to-end delay. Traditionally, routing schemes have been designed for specific applications and may not achieve the optimum network-wide performance in most kinds of network scenarios. In this paper, CRQ-routing has been shown to achieve network-wide performance in scenarios with various levels of channel dynamicity and unpredictability in regards to PU activities. Furthermore, CRQ-routing is simple to implement since it does not require additional hardware or geographical information. Basically, CRQ-routing is a Reinforcement Learning- (RL-) based approach that provides intelligence to SUs so that they can learn to make dynamic and efficient routing decisions on the fly while addressing the important characteristics of CRNs namely, dynamicity and unpredictability of channel availability, and SUs´ interference to PUs. Simulation results show that CRQ-routing reduces the interference to PUs for up to 43% compared to traditional routing schemes, and it achieves lower end-to-end delay under highly unpredictable PU activities.