Linear programming formulations of stochastic control problems

Stochastic control problems having infinite-horizon discounted, long-term average, finite horizon or first exit criteria are shown to be equivalent, under very general conditions, to linear programs over collections of measures satisfying an identity appropriate to the criterion. Existence of processes corresponding to the measures satisfying the various identities is established, which is necessary for the proof of equivalence. As a corollary, existence of solutions to the martingale problem corresponding to solutions of the forward equation for Markov processes is shown. Since every solution of the martingale problem corresponds to a solution of the forward equation, uniqueness of solutions of the forward equation is equivalent to uniqueness of the solutions of the martingale problem