Reachability analysis of dynamic programming based controlled systems

The design of feedback controllers based on the numerical solution of the dynamic programming equations leads to quasi-optimal system behaviour. The gap to the optimal behaviour depends on the accuracy of the numerical solution. As the numerical solution gets coarser, stability of the closed loop system also becomes a concern. This paper proposes a procedure for the reachability analysis of dynamic programming based controlled systems. The proposed verification procedure is based on the numerical solution of the dynamic programming equations (value function) and employs tools from constrained convex and semidefinite programming.