Task-Driven Learning of Spatial Combinations of Visual Features

Solving a visual, interactive task can often be thought of as building a mapping from visual stimuli to appropriate actions. Clearly, the extracted visual characteristics that index into the repertoire of actions must be sufficiently rich to distinguish situations that demand distinct actions. Spatial combinations of local features permit, in principle, the construction of features at various levels of discriminative power. We present an algorithm for selecting relevant spatial combinations of visual features by exercising a given task in a closed-loop learning process based on Reinforcement Learning. The algorithm operates by progressively splitting the perceptual space into distinct regions. Whenever the agent detects perceptual aliasing of distinct world states, it constructs a spatial combination of visual features that disambiguates the aliased states. We demonstrate the efficacy of our algorithm on a version of the classical "Car on the Hill" control problem where position and velocity are presented to the agent visually, in a way that the task is unsolvable using individual point features.