Object manipulation in visuo-haptic augmented reality with physics-based animation

We present an approach for visuo-haptic augmented reality, which is focused on object manipulation tasks. The interactive environment combines three-DOF haptic rendering and physics-based animation. A desktop haptic device with force feedback is driven by the user to interact with movable virtual objects that are superimposed upon a visual representation of the real workspace. Virtual objects coexist with real objects in the augmented reality space and are simulated in a physically plausible manner. The system supports both rigid and deformable objects with arbitrary shape. Accurate algorithms for camera calibration, registration, object manipulation and feedback computation have also been developed. Several experiments have been performed in order to test both single and dual-user collaborative tasks.