Title :
Decentralized learning for multi-player multi-armed bandits
Author :
Kalathil, Dileep ; Nayyar, Naumaan ; Jain, R.
Author_Institution :
Dept. of Electr. Eng., Univ. of Southern California, Los Angeles, CA, USA
Abstract :
We consider the problem of distributed online learning with multiple players in multi-armed bandit models. Each player can pick among multiple arms. As a player picks an arm, it gets a reward from an unknown distribution with an unknown mean. The arms give different rewards to different players. If two players pick the same arm, there is a “collision”, and neither of them get any reward. There is no dedicated control channel for coordination or communication among the players. Any other communication between the users is costly and will add to the regret. We propose an online index-based learning policy called dUCB4 algorithm that trades off exploration v. exploitation in the right way, and achieves expected regret that grows at most near-O(log2 T). The motivation comes from opportunistic spectrum access by multiple secondary users in cognitive radio networks wherein they must pick among various wireless channels that look different to different users.
Keywords :
game theory; learning (artificial intelligence); cognitive radio network; collision; dUCB4 algorithm; decentralized learning; distributed online learning; multiplayer multiarmed bandit model; online index-based learning policy; opportunistic spectrum access; player communication; player coordination; secondary user; user communication; wireless channel; Algorithm design and analysis; Cognitive radio; Indexes; Radiation detectors; USA Councils; Vectors; Distributed adaptive control; multi-agent systems; multiarmed bandits; online learning;
Conference_Titel :
Decision and Control (CDC), 2012 IEEE 51st Annual Conference on
Conference_Location :
Maui, HI
Print_ISBN :
978-1-4673-2065-8
Electronic_ISBN :
0743-1546
DOI :
10.1109/CDC.2012.6426587