Image-based haptic display via a novel pen-shaped haptic device on touch screens

Haptic interaction is a new interactive mode in the interaction technology between human and touch screens. In this work, we present an image-based haptic interaction system on touch screens. A novel haptic pen is designed for haptic display of visual image. It generates force feedback and tactile feedback through electromechanical structure and piezoelectric ceramics respectively. Haptic model of image is built by image processing and haptic rendering. In image processing, the image is decomposed into geometry and textures by local total variation for distinguishing contour shape from surface details. And the contour lines are extracted from the geometry based on the adaptive flow-based difference-of-Gaussians algorithm. Then the height information of image is recovered via shape from shading algorithm and expressed by electromechanical structure. Textures and contour lines are displayed by piezoelectric ceramics. Finally, the haptic perception experiment is conducted to investigate effect of perception with different haptic pens and modes of haptic interaction.