Downlink femto-to-macro interference management based on Fuzzy Q-Learning

This paper proposes a distributed solution for resource allocation in femtocell systems in order to control the downlink femto-to-macro aggregated interference. We propose a solution based on intelligent and self-organized femtocells implementing a decentralized Fuzzy Q-Learning (FQL), which with respect to other realtime multiagent Reinforcement Learning (RL) techniques allows to generalize the state space and to generate continuous actions, besides significantly speeding up the learning process. In particular, we propose a novel scheme able to maintain interference at the macrocell users below a threshold and at the same maximize the femtocells capacity, which was not considered by previous works of the same authors. We evaluate the FQL paradigm in the context of a 3rd Generation Partnership Project (3GPP) compliant Orthogonal Frequency Division Multiple Access (OFDMA) femtocell network.