Cell selection in two-tier femtocell networks with open/closed access using evolutionary game

Cell selection is an important issue in femtocell networks, which can balance the utilization of the whole network. In this paper, we investigate cell selection problem in a two-tier femtocell network that contains a micro base station (MBS) and several femtocells with different access methods and coverage areas. We propose the evolutionary game model to describe the dynamics of the cell selection process and consider the evolutionary equilibrium as the solution. In order to achieve the evolutionary equilibrium, we introduce the reinforcement learning algorithm that can help distributed individual users make selection decisions independently. With their own knowledge of the past, the users can learn to achieve the evolutionary equilibrium without complete knowledge of other users. Finally, the performance of the evolutionary game and reinforcement learning algorithm is analyzed, and simulation results show the convergence and effectiveness of the proposed algorithm.