Dynamic uplink/downlink configuration using Q-learning in femtocell networks

With increasingly popular services containing asymmetric uplink (UL) and downlink (DL) traffic, Time-Division Duplexing (TDD) shows more advantages than Frequency-Division Duplexing (FDD) in wireless cellular networks due to its flexibility of dynamic UL/DL configuration. A major technical challenge in TDD is how to optimize the UL/DL switching point for each individual cell to meet its asymmetric traffic demand without causing severe cross-link interference. In this paper, we formulate the dynamic UL/DL configuration problem in non-cooperative femtocell networks as a multi-agent Q-learning process. Each femto base station (FBS) is treated as an agent. It senses its local interference environment and then takes an action. The optimal UL/DL switching policy is learned gradually by each FBS through trial and error with the objective of minimizing the overall packet transmission time. The proposed Q-learning based method takes both the asymmetric traffic demands and the interference into consideration. Simulation results show that our proposed method can achieve higher cell average packet throughput than the existing queue-aware dynamic TDD method and the fixed TDD method.