A Q-learning based approach to interference avoidance in self-organized femtocell networks

The femtocell concept is an emerging technology for deploying the next generation of the wireless networks, aiming at indoor coverage enhancement, increasing capacity, and offloading the overlay macrocell traffic. Nevertheless, the detrimental factor in such networks is co-channel interference between macrocells and femtocells, as well as among neighboring femtocells. This in turn can dramatically decrease the overall capacity of the network. In addition, due to their non-coordinated nature, femtocells need to self-organize in a distributed manner not to cause interference on the macrocell, while at the same time managing interference among neighboring femtocells. This paper proposes and analyzes a Reinforcement-Learning (RL) framework where a macrocell network is underlaid with femtocells sharing the same spectrum. A distributed Q-learning algorithm is proposed in which each Femto Base Station/Access Point (FBS/FAP) gradually learns (by interacting with its local environment) through trials and errors, and adapt the channel selection strategy until reaching convergence. The proposed Q-learning algorithm is cast into high level and low level subproblems, in which the former finds in a decentralized way the channel allocation through Q-learning, while the latter computes the optimal power allocation. Investigations show that through learning, femtocells are not only able to self-organize with only local information, but also mitigate their interference towards the macrocell network.