Title :
Online Adaptive Optimization Algorithm for Semi-Markov Control Processes
Author :
Jiang Qi ; Xi Hongsheng ; Yin Baoqun
Author_Institution :
Dept. of Autom., Univ. of Sci. & Technol. of China, China
Abstract :
Semi-Markov control problems with unknown kernel are considered, a reinforcement learning based online adaptive optimization algorithm is proposed. First an event-driven stochastic switching model is introduced to formulate the semi-Markov control problems. Then by utilizing the features of event-driven policy an optimization algorithm that combines policy gradient estimation and stochastic approximation is derived. This algorithm can converge to global optimization without the explicit knowledge of the semi-Markov kernel. Moreover, this algorithm does not require the computation of performance potentials or other related quantities (e.g. Q-factors) and therefore saves computational cost significantly. Simulation results demonstrate the effectiveness of the proposed algorithm.
Keywords :
Markov processes; adaptive systems; gradient methods; learning (artificial intelligence); optimisation; time-varying systems; event-driven policy; event-driven stochastic switching; online adaptive optimization algorithm; policy gradient estimation; reinforcement learning; semiMarkov control processes; stochastic approximation; Adaptive control; Approximation algorithms; Computational efficiency; Computational modeling; Kernel; Learning; Process control; Programmable control; Q factor; Stochastic processes; Event-based optimization; Performance potential; Policy gradient estimation; Semi-Markov control processes; Stochastic approximation;
Conference_Titel :
Control Conference, 2006. CCC 2006. Chinese
Conference_Location :
Harbin
Print_ISBN :
7-81077-802-1
DOI :
10.1109/CHICC.2006.280525
