Neural network-based adaptive critic designs for self-learning control

We consider the implementation of adaptive critic designs using neural networks. The present scheme is within the general framework of approximate dynamic programming where optimal/suboptimal control is achieved through learning using multilayer feedforward neural networks. We will develop a class of adaptive critic designs that can be classified as (model-free) action-dependent heuristic dynamic programming (ADHDP). We believe that the present ADHDP is equivalent to the conventional model-based HDP since the model network in the latter can be viewed as completely embedded in the critic network.