Relay selection with no side information: An adversarial bandit approach

Multi-armed bandit games form a class of sequential optimization problems, in which a player sequentially pulls an arm, selected from a known and finite set of arms, in order to receive an a priori unknown reward. Since the player does not know the arm with the highest reward in advance, it utilizes a well-designed selection strategy to minimize the so-called regret, which, roughly speaking, results from the lack of this information. This paper studies cooperative transmission in a dense mobile network, where users compete for utilizing a number of relays to improve the quality of transmissions. Under the assumption of no side information available to the users, the relay selection and assignment problem is formulated as an adversarial multi-player multi-armed bandit game. Based on this formulation, a selection strategy is proposed that is shown to guarantee the convergence of the empirical frequencies of the game to a correlated equilibrium. Moreover, applying the experimental regret testing protocol shows that the empirical frequencies of the relay selection game converges to Nash equilibrium. Finally, experimental evaluations are carried out to compare the performance of various selection strategies and with it to demonstrate the effectiveness of the proposed approach. The proposed game model and selection strategies can be used in a wide range of wireless networking scenarios, such as spectrum pulling in cognitive radio networks and base station assignment in cellular networks.