DocumentCode :
3502327
Title :
Distributed Reinforcement Learning based MAC protocols for autonomous cognitive secondary users
Author :
Bkassiny, Mario ; Jayaweera, Sudharman K. ; Avery, Keith A.
Author_Institution :
Dept. of Electr. & Comput. Eng., Univ. of New Mexico, Albuquerque, NM, USA
fYear :
2011
fDate :
15-16 April 2011
Firstpage :
1
Lastpage :
6
Abstract :
We consider a decentralized cognitive radio network in which autonomous secondary users seek spectrum opportunities in licensed spectrum bands. We assume that the primary users´ channel occupancy follows a Markovian evolution, and formulate the spectrum sensing problem as a Decentralized Partially Observable Markov Decision Process (DEC-POMDP). We develop a distributed Reinforcement Learning (RL) algorithm that allows each autonomous cognitive radio to distributively learn its own spectrum sensing policy. The resulting decentralized sensing policy enables secondary users to non-cooperatively reach an equilibrium that leads to high utilization of idle channels while minimizing the collisions among secondary cognitive radios. Moreover, we propose a decentralized channel access policy that permits controlling, with high accuracy, the collision probability with primary users. Our numerical results validate the robustness of this collision probability control as the sensing noise changes. They also show the efficiency of the proposed learning algorithm in improving the spectrum utilization.
Keywords :
Markov processes; access protocols; cognitive radio; learning (artificial intelligence); probability; radio spectrum management; wireless channels; MAC protocols; Markov decision process; channel occupancy; collision probability; decentralized channel access; decentralized cognitive radio network; distributed reinforcement learning; licensed spectrum bands; spectrum sensing problem; spectrum utilization; Cognitive radio; Computational modeling; Detectors; Indexes; Learning; Markov processes; Cognitive radio; reinforcement learning;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Wireless and Optical Communications Conference (WOCC), 2011 20th Annual
Conference_Location :
Newark, NJ
Print_ISBN :
978-1-4577-0453-6
Type :
conf
DOI :
10.1109/WOCC.2011.5872298
Filename :
5872298
Link To Document :
بازگشت