Optimal regulation and reinforcement learning for the nonholonomic integrator

Reinforcement learning methods based on the Hamilton-Jacobi-Bellman equation have proven to be effective for linear systems. We consider the extension of these methods to a class of nonlinear systems whose linearizations are not controllable. Optimal values for a discounted, infinite horizon cost function based on a smooth homogeneous norm are proposed and validated both for the continuous-time and for the discrete-time three-dimensional nonholonomic integrator