Joint development of disparity tuning and vergence control

Behavior and sensory perception are mutually dependent. Sensory perception drives behavior, but behavior can also influence the development of sensory perception, by altering the statistics of the sensory input. Thus, there is a “chicken-and-egg” problem as to which arises first. We propose here a solution to this problem in the context of the neural processing of binocular disparity and the behavioral control of binocular vergence to maintain fixation. We show that it is possible for both the neural processing and the control policy to develop simultaneously. In particular, we assume that the neural processing develops through learning a sparse complex-cell representation of the input, and that the control policy simultaneously develops through reinforcement learning to maximize the activity in this complex cell representation. These processes are coupled. The control policy determines the statistics of the input, which determines the sparse coding that develops, which in turn determines the reward maximized by the control policy. Our experiments show that both disparity selective binocular receptive fields and a successful binocular fixation policy develop. Our results underline the importance of behavior, as we show that on the same input but in the absence of learned behavior, much fewer disparity selective binocular receptive fields develop.