Nearly optimal HJB solution for constrained input systems using a neural network least-squares approach

We consider the use of nonlinear networks towards obtaining nearly optimal solutions to constrained control problems. The method is based on least-squares successive approximation solution of the generalized HJB (Hamilton-Jacobi-Bellman) equation which appears in optimization problems. Successive approximation using the GHJB has not yet been applied for bounded controls. The proposed method successively solves the GHJB equation on a well-defined region of attraction making use of a suitable nonquadratic functional that allows us to work with smooth bounded controls. A neural network is used to approximate the GHJB solution. It is shown that the result is a closed-loop control based on a neural net that has been tuned a priori off-line. The control law structure is shown to have the largest possible region of asymptotic stability. As the order of the network is increased, and as the algorithm is run on more points in the well-defined region of attraction, it is shown that the network converges to the solution of the inherently nonlinear HJB equation associated with the bounded control.