A novel miniature multi-mode haptic pen for image interaction on mobile terminal

In this paper, a novel miniature multi-mode haptic pen is developed to interact with image on mobile terminal. The haptic pen can provide two types of vibrotactile feedback by linear resonant actuator (LRAs) and piezo-ceramic actuator. It can provide passive force feedback by a miniature magnetorheological (MR) fluids damper which is designed independently. In order to obtain the features of an image, the haptic interactive software is developed based on Android system. The software can extract the height of each pixel by the shape from shading (SFS) algorithm, and it can also detect the edges of an image by Sobel algorithm. When the haptic pen slides on an image, feature information of the image will be transmitted to the haptic pen via Bluetooth, including the height of each pixel, the edges of an image and the stiffness of virtual objects. Meanwhile, user will get multi-mode haptic sensation by different patterns of haptic display methods. Finally, three haptic interaction experiments are carried out to evaluate the performance of the haptic pen in displaying the height, contour and stiffness of an image.