Direct adaptive control of nonlinear systems using neural networks and stochastic approximation

The use of neural networks (NNs) in controlling a nonlinear stochastic system with unknown process equations is considered. The NN is used to model the resulting unknown control law. The approach is based on using the output error of the system to train the NN controller without the need to construct a separate model (NN or other type) for the unknown process dynamics. To implement such an approach, it is required that connection weights in the NN be estimated while the system is being controlled. As a result of the feedback of the unknown process dynamics, however, it is not possible to determine the gradient of the loss function for use in standard (backpropagation-type) weight estimation algorithms. Therefore, the authors consider the use of a new stochastic approximation algorithm for this weight estimation that is based on a simultaneous perturbation gradient approximation that only requires the system output error. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations