Multi-sensor and prediction fusion for contact detection and localization

Robot perception of physical interaction with the world can be achieved based on different sensory modalities: tactile, force-torque, vision, laser, sonar, proprioception, accelerometers, etc. An important problem and research topic in robotics is the question of how to fuse multiple sensory modalities to provide the robot with advanced perception capabilities. However, in the context of contact localization in grasping and manipulation tasks, the fusion of multiple sensory information has not been addressed so far. We propose a sensory information fusion approach for contact detection and localization. The approach relies on the generation of contact hypotheses and the fusion of these hypotheses to determine the likelihood of a contact at a certain location leading to an improved robustness and precision of contact detection. In addition, the approach allows the integration of multiple sensors, environment, context and predictions. We have implemented the proposed approach on two dual-arm robots and validated it through several experiments.