Reinforcement learning based spectrum-aware routing in multi-hop cognitive radio networks

Routing in multi-hop cognitive radio networks (CRN) should be spectrum-aware. In this paper, two adaptive reinforcement learning based spectrum-aware routing protocols are introduced. Q-learning and dual reinforcement learning are applied respectively for them. The cognitive nodes store a table of Q values that estimate the numbers of available channels on the routes and update them while routing. So they can adaptively learn good routes which have more available channels from just local information. Compared to the previous spectrum aware routing protocols in multi-hop cognitive radio networks, they are simpler and easier to implement, more cost-effective, and can avoid drawbacks in on-demand protocols but still keep adaptive and dynamic routing. Both of our protocols perform better than the spectrum-aware shortest path protocol when network load is not too low. In the meantime, spectrum-aware DRQ-routing learns the optimal routing policy 1.5 times as fast as the spectrum-aware Q-routing at low and medium network load. It also learns a routing policy which is more than seven times as good as that of spectrum-aware Q-routing at high network load.