Emergence of delayed reward learning from sensorimotor coordination

When building autonomous robotic agents, designers almost always make assumptions about how the control system relates sensory information to motor actions, in order to provide the agent with a set of basic behaviors. This raises the question of how arbitrary these assumptions are, and to what extent the introduced biases reduce the potential for the generation of new behaviors arising from interaction with the environment. In this paper, we propose a new model of robot control architecture, consisting merely of homogeneous, non-hierarchical sensorimotor coupling. We show that a robot using this model can display coherent complex behaviors which emerge from the agent-environment interaction, such as tracking an object, or solving a task based on the temporal relationship between an early clue and a delayed reward.