Reinforcement learning application scenario for Opportunistic Spectrum Access

We tackle in this work a concrete scenario that illustrates the behavior of Cognitive Radio equipment within an Opportunistic Spectrum Access context. We assume that there exist two sets of users. On the one hand Primary Users who own the spectrum pool of interest. And one the other hand, Secondary Users who aim at exploiting vacant communication opportunities left by Primary Users at a given time in a given band. Moreover, Secondary Users are assumed to have no a priori knowledge on Primary Users behavior and aim at learning missing information while exploiting found communication opportunities. For that purpose, we model Primary Users´ frequency bands occupations pattern using an OFDM modulation. While, the introduced secondary user´s learning process relies on an energy detector and a reinforcement learning algorithm known as UCB₁. The complete model is developed on Simulink to illustrate the behavior of the Primary Network as well as the Secondary User.