Sample-based potentials estimation for the optimal control of stochastic system

An optimization method based on perturbation analysis is applied to stochastic system. A policy iteration approach is designed by the performance sensitivity formula which is constructed with potentials. For estimating the potentials, the Poisson equation is viewed as a system of linear equation, then a least squares policy evaluation method is adopted, and the selection of basis function is also discussed for getting a better performance of approximation. The simulation shows the effectiveness of the policy iteration and the approximation approach.