A QoS aware reinforcement learning algorithm for macro-femto interference in dynamic environments

Network operators are considering femtocell solutions as a mean of reducing costs, offloading their networks and increasing profits but this at the same time gives rise to new technological challenges. The most stringent ones are the resources that the femtocells should be using and the interference between the macro and the femto layers. We propose an unsupervised learning algorithm, namely reinforcement learning as the means of achieving self-organizing capabilities for the femtocells. The proposed algorithm is characterized by femto-QoS awareness, as the actions of the femtocells are directed at avoiding interference on the macro layer but at the same time achieving the QoS requirements of the femtocells. Dealing with OFDMA based systems, different service classes for both the macro and the femto users are envisioned and equal priority is assigned among users. The different QoS requirements allow applying a Markov chain prediction on the number of resources required by a femtocell, enhancing the performance of the algorithm. Finally, the proposed algorithm is tested in a highly dynamic environment in which the allocation of resources to the macro users can change at each time step.