Adaptive optimal control algorithm for continuous-time nonlinear systems based on policy iteration

In this paper we develop a new online adaptive control scheme, for partially unknown nonlinear systems, which converges to the optimal state feedback control solution for affine in the inputs nonlinear systems. The derivation of the optimal adaptive control algorithm is presented in a continuous-time framework. The optimal control solution will be obtained in a direct fashion, without system identification. The algorithm is an online approach to policy iterations based on an adaptive critic structure to find an approximate solution to the state feedback, infinite-horizon, optimal control problem.