Vision-tactile-force integration and robot physical interaction

This paper presents an approach for integrating vision, tactile and force sensors in a robotic manipulation framework. Having an initial estimation of the object pose in the environment, a position-based visual servoing loop controls the hand for task execution, based on the input received from a model-based articular pose estimator following the Virtual Visual Servoing approach. The visual control is combined with another control signal obtained from tactile feedback, through a set of selection matrices that can be modified at runtime in order to select the best modality for a given cartesian degree of freedom. The result of the preliminary integration is modified by an impedance force controller, in charge of performing the task motion along the task direction, at the same time that forces are regulated on the rest of directions. The design of the controller allows to perform the task even if a sensor is not available or provides inaccurate data. Several experiments are performed, first by considering only force feedback, and then adding vision and, finally, tactile information. Errors in the estimation of the object initial position are manually introduced in the experiments, and results show how the vision-tactile-force combination is able to deal with them, performing much better than the vision-force and force-alone approaches.