Numerical Approximations for Optimal Controls for Stochastic Systems With Delays

The Markov chain approximation numerical methods are widely used to compute optimal value functions and controls for stochastic and deterministic systems. We extend them to controlled general nonlinear delayed reflected diffusion models. The path, control, and reflection terms can all be delayed. Previous work developed convergent numerical approximations. But when the control and reflection terms are delayed there are impossible demands on memory. An alternative "dual" approach was proposed by Kwong and Vintner for the linear deterministic system with a quadratic cost function. We extend it to the general nonlinear stochastic system, develop the Markov chain approximations and numerical algorithms, and outline the convergence theorems. The approach reduces the memory requirement significantly