Robust asymptotic tracking of a class of nonlinear systems using an adaptive critic based controller

Traditional Reinforcement Learning (RL) controllers are based on a discrete formulation of the Dynamic Programming (DP) problem, which impedes the development of rigorous stability analysis of continuous-time closed loop controllers for uncertain nonlinear systems. Non-DP based RL controllers typically yield a uniformly ultimately bounded (UUB) stability result due to the presence of disturbances and unknown approximation errors. In this paper a non-DP based reinforcement learning scheme is developed for asymptotic tracking of a class of uncertain nonlinear systems with bounded disturbances. A recently developed RISE (Robust Integral of the Sign of the Error) feedback technique is used in conjunction with a feedforward neural network (NN) based Actor-Critic architecture to yield a semi-global asymptotic result. A composite weight tuning law for the Action NN, consisting of both unsupervised and reinforcement learning terms, is developed based on Lyapunov stability analysis.