Markov chain approximations for deterministic control problems with affine dynamics and quadratic cost in the control

We restrict our attention to an important class of deterministic optimal control problems: those with dynamics that are affine in the control and cost that is quadratic in the control. As a representative control problem in this class we consider an escape time problem. The chains we construct are much more broadly applicable, since the only germane part of the problem formulation will be the form of the dynamics and the running cost. In particular, these chains can also be used for discounted problems, infinite time problems that do not involve discounting and constrained dynamics