Perception of sound renderings via vibrotactile feedback

The vibration behavior of acoustic music instruments, as perceived through touch, is known to play an important role in the interaction between performers and music instruments. This research explores the relevance and utility of including a tactile simulation of this behavior in computer music interfaces, in a snapshot taken during early learning, and including tradeoffs of using this channel for other information. Vibrotactile (VT) renderings of different types of sounds were presented to subjects via a handle fitted with vibration actuators. Subjects (1) evaluated consistency of the sound with the VT rendering, (2) identified from a “lineup” the VT rendering that did not match a sound, and (3) provided data on how inclusion of VT cues in a rhythm-tapping task affected perception quality of the VT rendering. For the last, a distractor task allowed us to measure both the usability of the cues and the degradation in perception of the VT rendering. All renderings were either played directly through vibration transducers, or first altered by modifying the sound´s frequency content. VT playback was delivered at intensities similar to those experienced in traditional acoustic music instruments. Subjects indicated that specific alterations to the original, direct VT rendering were consistent with the source sound more often than the original was, and were able to differentiate between correct and incorrect renderings of a source sound. However, the latter ability was masked when subjects were provided with and able to effectively utilize extra VT cues added to the VT feedback to improve their performance, suggesting that the VT cues were of greater utility than the VT mimicking. We discuss the relevance of these findings on the design of computer music interfaces.