MS-Taxonomy: a conceptual framework for designing multi-sensory displays

One important task that the designer of multi-sensory displays must perform is the comparison of displays designed for the different senses. Comparing designs for different visualisations is difficult enough but comparing a visual display with an auditory display, or haptic display is harder still. Yet the designer of multi-sensory displays would like to make sensible decisions about when to use each modality. This work describes the MS-Taxonomy, a classification of abstract data displays that is general for all senses. This allows the same terminology to be used for describing visualisation, sonification and haptic display designs. The classification of displays is hierarchical and describes multiple levels of abstraction. In software engineering terms the taxonomy allows a designer to consider reuse at both an abstract architectural level and also a more detailed component level. Thus design mappings can be discussed independently of the sensory modality to be used. This allows for exactly the same design to be implemented for each sense and subsequently compared or transferred between modalities.