Application of Dynamic Programming to Singular Optimal Control Problems

The difficulty in obtaining the optimal control policy, where singular subarcs appear when Pontryagin´s Maximum Principle is applied, does not arise if dynamic programming is used. To apply dynamic programming, a relatively coarse grid is used for the state variables and only a small number of allowable values of control are taken at each time stage. The grid and the range for allowable control are contracted after every iteration to provide an accurate control policy after a sufficient number of iterations. The effects of the coarseness of the grid and the size of the reduction factor are examined with a typical chemical engineering problem that has been found to be difficult to solve by other means. To reduce computational effort for high-dimensional systems, it is recommended to use for the grid elements only accessible states. The choice of such grid elements is examined by considering an example of higher dimension. The proposed procedure is easy to program and yields better results than have been reported in the literature for the systems considered.