Blind in a virtual world: Using sensory substitution for generically increasing the accessibility of graphical virtual environments

Graphical virtual environments are currently far from accessible to the blind as most of their content is visual. While several previous environment-specific tools have indeed increased accessibility to specific environments they do not offer a generic solution. This is especially unfortunate as such environments hold great potential for the blind, e.g., for safe orientation and learning. Visual-to-audio Sensory Substitution Devices (SSDs) can potentially increase their accessibility in such a generic fashion by sonifying the on-screen content regardless of the specific environment. Using SSDs also taps into the skills gained from using these same SSDs for completely different tasks, including in the real world. However, whether congenitally blind users will be able to use this information to perceive and interact successfully virtually is currently unclear. We tested this using the EyeMusic SSD, which conveys shape and color information, to perform virtual tasks otherwise not possible without vision. We show that these tasks can be accomplished by the congenitally blind.