Finite horizon discrete-time approximate dynamic programming

Dynamic programming for discrete time system is difficult due to the "curse of dimensionality": one has to find a series of control actions that must be taken in sequence, hoping that this sequence will lead to the optimal performance cost, but the total cost of those actions will be unknown until the end of that sequence. In this paper, we present our work on adaptive optimal control of nonlinear discrete time system using neural networks. We study the relationships of optimal controls for different control steps and then develop a neural dynamic programming algorithm based on these relationships