The efficient solution of the (quietly constrained) noisy, linear regulator problem

In a previous paper we gave a new, natural extension of the calculus of variations/optimal control theory to a (strong) stochastic setting. We now extend the theory of this most fundamental chapter of optimal control in several directions.Most importantly we present a newmethod of stochastic control, adding Brownian motion which makes the problem “noisy.” Secondly, we show how to obtain efficient solutions: direct stochastic integration for simpler problems and/or efficient and accurate numerical methods with a global a priori error of O(h3/2) for more complex problems. Finally, we include “quiet” constraints, i.e. deterministic relationships between the state and control variables. Our theory and results can be immediately restricted to the non “noisy” (deterministic) case yielding efficient, numerical solution techniques and an a priori error of O(h2). In this event we obtain the most efficient method of solving the (constrained) classical Linear Regulator Problem. Our methods are different from the standard theory of stochastic control. In some cases the solutions coincide or at least are closely related. However, our methods have many advantages including those mentioned above. In addition, our methods more directly follow the motivation and theory of classical (deterministic) optimization which is perhaps the most important area of physical and engineering science. Our results follow from related ideas in the deterministic theory. Thus, our approximation methods follow by guessing at an algorithm, but the proof of global convergence uses stochastic techniques because our trajectories are not differentiable. Along these lines, a general drift term in the trajectory equation is properly viewed as an added constraint and extends ideas given in the deterministic case by the first author. © 2006 Elsevier Inc. All rights reserved.