Suboptimal control for nonlinear stochastic systems

Theoretical procedures are developed for comparing the performance of arbitrarily selected admissible feedback controls among themselves with that of the optimal solution of a nonlinear optimal stochastic control problem. Iterative design schemes are proposed for successively improving the performance of a controller until a satisfactory design is achieved. Specifically, the exact design procedure is based on the generalized Hamilton-Jacobi-Bellman equation for the value function of nonlinear stochastic systems, and the approximate design procedure for the nonlinear stochastic regular problem with an infinite horizon is developed by using the upper and lower bounds to the value functions. For a given controller, both the upper and lower bounds to its value function can be obtained by solving a partial differential inequality. In particular, the upper and lower bounds to the optimal value function, which may be used as a measure to evaluate the acceptability of suboptimal controllers, can be constructed without actually knowing the optimal controller