Stochastic optimal control subject to variational norm uncertainty: Dynamic programming and HJB equation

This paper is concerned with the application of recent results in optimization of stochastic uncertain systems on general abstract spaces, when the uncertain probability measure of the system is described by a ball with respect to the variational norm, centered at the nominal measure having certain radius. The pay-off is a linear functional of the uncertain measure. The maximization of the linear functional over the uncertainty ball is shown to be equivalent to an optimization of a convex combination of L₁ and L_∞ norms. Further, the maximizing measure is constructed from the family of tilted exponential probability measures. The abstract results are applied to uncertain continuous-time nonlinear stochastic controlled systems, in which the control seeks to minimize a linear functional while the measure seeks to maximize it over the variational norm uncertainty set. By invoking the maximizing measure obtained at the abstract setting, the resulting pay-off which is minimized over the admissible controls is a non-linear functional of the nominal measure. The resulting non-standard optimal control problem is addressed by deriving a new type of principle of optimality, while dynamic programming is employed to derive a generalized Hamilton-Jacobi-Bellman (HJB) equation satisfied by the value function.