Touching sounds: audio virtual surfaces

This prospective study concerning the perception of audio virtual surfaces (AVSs) was inspired by two different research fields: sensory substitution and haptic and touch perception. We define an Audio Virtual Surface as a region of space that triggers sounds when the user touches it or moves into it. First, we describe an example of interactive setup using an AVS to simulate a sonic interaction with a virtual water tank. Then, we present an experiment designed to investigate the ability of blindfolded adults to discriminate between concave and convex AVSs using only the gesture-sound interaction. Two groups received different sound feedback, a static one indicating presence in the AVS, and a static+dynamic one (related to the component of the hand velocity tangential to the surface). In order to demonstrate that curvature direction was correctly perceived, we estimated their discrimination thresholds with a psychophysical staircase procedure. Results show that most of the participants were able to learn the task. The best results were obtained with the additional dynamic feedback. Gestural patterns emerged from the interaction, suggesting the use of auditory representations of the virtual object. This work proposes a contribution to the introduction in Virtual Reality of sonic interactions with auditory virtual objects. The setups we present raise new questions at both experimental (sensory substitution) and application levels (design of gesture-sound interaction for virtual reality).