Tractable Algorithm for Open Loop Stochastic Control

We present a fast numerical algorithm for open loop control of stochastic differential equations (SDEs). The major difference between this approach and standard stochastic control is that the control process is a deterministic function of time, and not a fully stochastic adapted process. This results in a tremendous gain in computational tractability: the complexity of our algorithm scales as the complexity of optimal control of ordinary differential equations. The algorithm computes the gradient of the cost function with respect to the control function via Monte Carlo sampling of the pathwise gradient. The pathwise gradient is itself computed by solving a set of forward-backward discretized SDEs. Two types of applications are investigated: stochastic system identification and parametric closed loop design