Representing normal and shearing forces on the mobile device with visuo-haptic interaction and a rear touch interface

The main contribution of this paper is to realize a visuo-haptic system that represents normal and shearing forces on a mobile device using visuo-haptic interaction and a rear-touch interface. Input with a touch-panel interface is typically inaccurate because a user cannot obtain haptic feedback, and the touch point is occluded by the user´s finger. To overcome this problem, we propose a visuo-haptic system with a rear-touch interface to evoke the haptic sensation without using haptic devices. For normal force, the system modifies the shape of a virtual object when the user presses on the device from behind. For shearing force, the system produces a mismatch between the speed of the touch point on the rear side, which is hidden by the monitor, and a cursor, which shows a touch point displayed on the monitor. Using this system, we conducted experiments to explore the effect of visuo-haptic interaction of normal and shearing forces with a rear-touch interface. The results demonstrated that more than 80% of participants perceived greater stiffness with the deformed model than the model not deformed by normal force. In addition, the results showed significant differences in perception of shearing force based on differing strengths of visual feedback.