Admission control by MDP theory: a single-sample-path-based approach

The theory of Markov decision process (MDP) has been widely applied to the networking management such as routing and admission control. However, the traditional MDP approach is mainly hindered by prohibitive computational complexity. The performance potential theory offers an efficient solution to alleviate such difficulties in infinite-horizon MDP problems. The concept “potential” leads to some important properties that allow it to be measured on a single sample path, thereby adapting to the dynamic characteristics in realistic applications such as high speed networks. In this paper, we investigate the application of single-sample-path-based potential theory to the admission control in the network with multiple classes of traffic. Optimal policies under different traffic characteristics are obtained with a fast convergence. Some simple and efficient algorithms are developed for online implementation