Packet Scheduling in Broadband Wireless Networks Using Neuro-Dynamic Programming

The design of future-generation broadband wireless network introduces a set of challenging technical issues. This paper focuses on the packet scheduling algorithms. The key difficulty of the problem lies in the high variability of wireless channel capacity and the unknown model of packet arrival process. We view the packet scheduling problem as a semi-Markov decision process (SMDP), and approximately solve the problem by using the methodology of neuro-dynamic programming (or reinforcement learning). The proposed algorithm, called neuro-dynamic programming scheduling (NDPS), employs a feature-based linear approximating architecture to produce a near optimal solution of the corresponding SMDP problem. Simulation experiment is carried out to demonstrate that NDPS can simultaneously achieve three performance objectives: (i) QoS differentiation and guarantee, (ii) high bandwidth utilization, and (iii) both short-term and long-term fairness.