Online critic-identifier-actor algorithm for optimal control of nonlinear systems

In this paper, a novel critic-identifier-actor optimal control scheme is designed for discrete-time affine nonlinear systems with uncertainties. A neural identifier is established to learn the unknown control coefficient matrix for affine nonlinear system working together with an actor-critic-based scheme to solve the optimal control in online and forward-in-time manner without value or policy iterations. A critic network learns approximate value function at each step. Another actor network attempts to improve the current policy based on the approximate value function. The weights of all neural networks (NNs) are updated at each sampling instant. Lyapunov theory is utilized to prove the stability of the closed-loop system. A simulation example is provided to illustrate the effectiveness of the developed method.