Performance analysis of Reinforcement Learning for achieving context-awareness and intelligence in Cognitive Radio networks

Cognitive radio (CR) is a novel and promising paradigm for next-generation wireless communication. It is able to sense and change its transmission and reception parameters adaptively according to spectrum availability at different channels. The cognition cycle (CC) is a state machine that is embodied in each CR that defines the mechanisms related to achieving context-awareness and intelligence including observation, orientation, learning, planning, decision making, and action selection. The CC is the key element in the design of various schemes in CR networks such as dynamic channel selection (DCS), scheduling and congestion control. Hence, a good implementation of the CC is of paramount importance. In this paper, reinforcement learning (RL) is employed to implement the CC. The main focus is to analyze the performance of RL as an approach to achieving context-awareness and intelligence in regard to DCS. The contributions of this paper are twofold. Firstly, we seek to justify whether RL is an appropriate tool to implement the CC. Secondly, we seek to understand the effects of changes on RL parameters on network performance. In addition, we propose solutions for the problems associated with the application of RL in DCS. The results presented in this paper show that RL is a promising approach.