Learning to Steer on Winding Tracks Using Semi-Parametric Control Policies

We present a semi-parametric control policy representation and use it to solve a series of nonholonomic control problems with input state spaces of up to 7 dimensions. A nearest-neighbor control policy is represented by a set of nodes that induce a Voronoi partitioning of the input space. The Voronoi cells then define local control actions. Direct policy search is applied to optimize the node locations and actions. The selective addition of nodes allows for progressive refinement of the control representation. We demonstrate this approach on the challenging problem of learning to steer cars and trucks-with-trailers around winding tracks with sharp corners. We consider the steering of both forwards and backwards-moving vehicles with only local sensory information. The steering behaviors for these nonholonomic systems are shown to generalize well to tracks not seen in training.