Delay aware cell association and user scheduling in heterogeneous overlay networks

In this paper, we propose a two timescale delay aware cell association and user scheduling design for downlink heterogeneous overlay networks. The cell association is adaptive to the global queue state information(QSI) over a longer timescale while the user scheduling is adaptive to both intra-cell queue state information(QSI) and intra-cell channel state information(CSI) and computed locally at each base stations(BSs) over a shorter timescale. The delay aware control is formulated as an infinite horizon partially observed Markov decision process(POMDP) and an equivalent Bellman equation in terms of cell association pattern Q-factor is derived. To reduce the computational complexity and facilitate distributed implementation, the cell association pattern Q-factor is approximated by the sum of per cell potential function, which can be further decomposed under specific assumption. After that, a novel distributed learning algorithm for potential function is proposed. The simulation results show that the proposed design outperforms the traditional cell association and user scheduling schemes and the learning algorithm converges surely.