Dynamic programming equations for constrained stochastic control

It is demonstrated that the dynamic programming approach provides a simple and versatile means for analyzing constrained stochastic control problems. Specifically, three such problems are analyzed using this approach. The problems analyzed are the problem of minimizing a discounted cost infinite horizon expectation objective subject to an identically structured constraint, the problem of minimizing a discounted cost infinite horizon minimax objective subject to a discounted expectation constraint, and the problem of minimizing a discounted expected cost objective subject to a minimax constraint. Using a dynamic programming approach, optimality equations are obtained for these problems. The optimality equations derived for the first two problems are apparently novel. Existence and uniqueness of solutions to the dynamic programming equations for the discounted cost infinite horizon problems are explicitly shown using the Banach fixed point theorem. Thus, the paper demonstrates that the dynamic programming theory for unconstrained stochastic control problems can be extended in a direct way to constrained stochastic control problems. The theory developed is illustrated by numerically solving the constrained stochastic control dynamic programming equations derived for simple example problems.