Dynamic channel selection and routing through reinforcement learning in Cognitive Radio Networks

Recent exploration in Cognitive Radio Network proved itself as emerging paradigm to attempt the underutilization of wireless spectrum. Routing is challenging problem due to intermittent spectrum availability and incomplete knowledge of environment. This paper proposes reinforcement learning based combined framework of channel selection and routing for multi-hop cognitive radio network. Reinforcement learning is generic method for resource utilization in a partially observable and non-stationary environment. In this paper, channel selection and routing is modeled as Markov Decision Process to design the methodology of learning the best resource allocation policies adopted in the process state, based on the feedback received from the environment. First the design of the reward, transition and value function is described which helps in evolving the policy for selecting channel which results in increased spectrum utilization. The routing strategy is described which is exploring different state-action pair to come up with various routing solution which are ranked according to their reinforcement signal. Overhead of rerouting is also minimized by providing backup routes. Agent experiences in the form of reinforcement signal can be used by each cognitive node to further refine the routing strategies.