Multimodal sensory integration for estimating rigid body motion of a dynamically manipulated object

In this paper, we implement a new rigid body estimation scheme combining the visual and tactile as well as finger tip force/torque sensory data. Construction of such a state estimator is motivated by the dynamic dexterous manipulation tasks which require accurate information about the motion of the target object in realtime. Visual information provides the absolute information of the position and orientation of the object but the measurement is often slow with computation delay. On the other hand, the tactile, or force/torque data can be acquired at a fast sample rate but often are corrupted by large amount of noise. Therefore, combination of these two sensing modalities can compensate for each other´s drawbacks. Mathematically, the estimator takes the form of a model-based multirate stochastic estimator with an additional algorithm to compensate for the measurement delay. The benefit of the proposed scheme is validated by simulation results using rolling contact model developed to represent an experimental setup currently being developed.