Stiffness modulation for Haptic Augmented Reality: Extension to 3D interaction

Haptic Augmented Reality (AR) allows a user to touch a real environment augmented with synthetic haptic stimuli. For example, medical students can palpate a virtual tumor inside a real mannequin using a haptic AR system to practice cancer detection. To realize such functionality, we need to alter the haptic attributes of a real object by means of virtual haptic feedback. Previously, we presented a haptic AR system with stiffness as a goal modulation property, and demonstrated its competent physical and perceptual performances for 1D interaction. In this paper, we extend the system so that a user can interact with a real object in any 3D exploratory pattern while perceiving its augmented stiffness. A series of algorithms are developed for contact detection, deformation estimation, force rendering, and force control. Their performances are thoroughly evaluated with real samples. A particular focus has been on minimizing the amount of preprocessing such as geometry modeling. Our haptic AR system can provide convincing stiffness modulation for real objects of relatively homogeneous deformation properties. The limitations of our AR system are also discussed along with a plan for future work.