Neural network-based finite-horizon approximately optimal control of uncertain affine nonlinear continuous-time systems

This paper develops a novel neural network (NN) based finite-horizon approximate optimal control of nonlinear continuous-time systems in affine form when the system dynamics are complete unknown. First an online NN identifier is proposed to learn the dynamics of the nonlinear continuous-time system. Subsequently, a second NN is utilized to learn the time-varying solution, or referred to as value function, of the Hamilton-Jacobi-Bellman (HJB) equation in an online and forward in time manner. Then, by using the estimated time-varying value function from the second NN and control coefficient matrix from the NN identifier, an approximate optimal control input is computed. To handle time varying value function, a NN with constant weights and time-varying activation function is considered and a suitable NN update law is derived based on normalized gradient descent approach. Further, in order to satisfy terminal constraint and ensure stability within the fixed final time, two extra terms, one corresponding to terminal constraint, and the other to stabilize the nonlinear system are added to the novel update law of the second NN. No initial stabilizing control is required. A uniformly ultimately boundedness of the closed-loop system is verified by using standard Lyapunov theory.