Machine Learning Based Knowledge Acquisition on Spectrum Usage for LTE Femtocells

The decentralised and ad hoc nature of femtocell deployments calls for distributed learning strategies to mitigate interference. We propose a distributed spectrum awareness scheme for femtocell networks, based on combined payoff and strategy reinforcement learning (RL) models. We present two different learning strategies, based on modifications to the Bush Mosteller (BM) RL and the Roth-Erev RL algorithms. The simulation results show the convergence behaviour of the learning strategies under a dynamic robust game. As compared to the Bush Mosteller (BM) RL, our modified BM (MBM) converges smoothly to a stable satisfactory solution. Moreover, the MBM significantly reduces the interference collision cost during the learning process. Both the MBM and the modified Roth-Erev (MRE) algorithms are stochastic-based learning strategies which require less computation than the gradient follower (GF) learning strategy and have the capability to escape from suboptimal solution.