Equivalent stochastic and deterministic optimal control problems

This paper presents a method of converting a stochastic optimal control problem into a deterministic one, where probability distributions on the state space of the stochastic model are taken as states in the deterministic model. Relations between policies and rewards of the two models are given and counterparts to known deterministic results are developed for the stochastic model. A novel feature of our framework is the enlargement of the set of admissible control laws to include universally measurable policies rather than only Borel measurable policies. This feature together with a new measurable selection theorem allow us to dispense with the notions of p-optimality of Blackwell and Strauch [11,6] and the notion of p-essential infimum of Striebel [12], and obtain all results in a form comparable to that for deterministic problems.