Online policy iteration algorithm for semi-Markov switching state-space control processes

An event-based online policy iteration algorithm is presented for addressing hierarchical optimization problems. First, an event-driven analytical model with dynamic hierarchy called semi-Markov switching state-space control processes is introduced. Then, by exploiting the structure of dynamic hierarchy and the features of event-driven policy, an online adaptive optimization algorithm that combines potentials estimation and policy iteration is proposed. The convergence of this algorithm is also proved. Finally, as an illustrative example, the dynamic service composition in a service overlay network is formulated and addressed. Simulation results demonstrate the effectiveness of the presented algorithm.