Optimization of Semi-Markov Switching State-space Control Processes for Network Communication Systems

Motivated by optimization of network communication systems, this paper presents an event-driven semi-Markov switching state-space control process with hierarchical dynamic architectures. First, the semi-Markov kernel of the switching control process is constructed, and the sensitivity formula for performance derivatives under average criterion is derived. Then, an online optimization algorithm that combines policy gradient estimation and stochastic approximation is proposed. This analytic model is with constructional flexibility and scalability, and the proposed optimization algorithm is adaptive and with less computational cost. Finally, as an illustrative example, the load balancing problem in a streaming media server cluster is formulated and addressed.