Visuo-haptic sensor for force measurement and contact shape estimation

Systems interacting with objects in unstructured environments require both haptic and visual sensors to acquire sufficient scene knowledge. Typically, separate sensors and processing systems are used for the two modalities. We propose to acquire haptic and visual measurements simultaneously, with the same standard camera that already observes the scene. The compression of a passive, deformable foam rod mounted on the actuator is measured visually, yielding a 1D stress function sampled along the contour of the rod. Like that, visual and haptic measurements are naturally coherent, the system is passively compliant and the complexity of the sensor subsystems is reduced. The approach is implemented and validated on a small robotic platform which is used for haptic exploration and interacts with objects by pushing them. In our experiments, friction forces and contact points are determined when the robot comes into contact with one or multiple obstacles. Furthermore, geometric primitives of their footprints are estimated. Using a simple exploration scheme, a haptic map of static and movable objects is built.