Decentralized Q-learning of LTE-femtocells for interference reduction in heterogeneous networks using cooperation

Femtocells have become an attractive approach for operators to offer extended services on their licensed UMTS/LTE spectrum. They are typically deployed indoors to improve coverage and provide high data rates. As a drawback, femtocells may cause interference to other femtocells or to the macrocellular wireless network, especially in co-channel closed access deployment scenarios. In order to reduce this downlink interference, we propose a solution based on intelligent and self-organized femtocells implementing decentralized Q-learning algorithms. We introduce a criterion to measure the expertness of the femtocells in order to enable cooperation to those femtocells that are activated after a certain time. Newly activated and non-experienced femtocells measure the expertness of neighbour femtocells, assign a weight to their knowledge, and learn from them accordingly. System level simulation results demonstrate the effectiveness of the proposed approach.