Sensor-fusion in spiking neural network that generates autonomous behavior in real mobile robot

We here introduce a novel adaptive controller for autonomous mobile robot that binds N types of sensory information. For each sensory modality, sensory-motor connection is made by a three-layered spiking neural network (SNN). The synaptic weights in the model have the property of spike timing-dependent plasticity (STDP) and regulated by presynaptic modulation signal from the sensory neurons. Each synaptic weight is incrementally adapted depending upon the firing rate of the presynaptic modulation signal and that of the hidden-layer neuron(s). Information from different types of sensors are bound at the motor neurons. A real mobile robot Khepera with the SNN controller quickly adapted into an open environment and performed the desired task successfully. This approach could be applicable to a robot with inputs of various sensory modalities and various types of motor outputs.