Spectrum allocation based on Q-Learning algorithm in femtocell networks

Interference is the main problem in femtocell networks. In this paper, a new dynamic spectrum allocation scheme for femtocell networks in OFDM scenario based on reinforcement learning is presented. The proposed algorithm allocates spectrum through Q-Learning to dynamically adjust the number of subchannels by different frequency reuse factors. The greater the reuse factor is, the fewer the shared subchannels in adjacent femtocells are. The reward function of Q-Learning considers spectral efficiency of all femtocells to ensure the minimum spectral efficiency of each cell as much as possible. Simulation results show that the proposed algorithm has a better system spectral efficiency with edge spectral efficiency guaranteed in comparison with other algorithms.