Context-dependent dynamic weighting of information from multiple sensory modalities

A major problem for the application of sensorimotor approaches to robot control is the classification of states. The typically immense sizes of sensorimotor state spaces render it very unlikely that exactly the same states are visited by the robot several times. In order to learn about the consequences of alternative behaviors in these states, a classification of similar or related states is necessary. This requires a metric to measure similarity between states. Under the premise that the robot should maximize its fitness, we studied the correlations between sensory data in different modalities and fitness values. We found that this correlation structure can serve as a context-dependent weighting of the importance of individual sensory channels that allows to define such a metric. In a collision-avoidance scenario we demonstrate that this results in (i) faster learning of successful actions, (ii) an acquired differentiation between sensory modalities, (iii) the possibility to use the full sensors resolution without quantization or compression, and (iv) a means to enhance sensor failure resilience.