Multisensory perception: beyond the visual in visualization

Mapping information onto more than one sensory modality might let us increase human bandwidth for understanding complex, multivariate data. Researchers have done significant work to explore the effective use of a single nonvisual sense for data display. Unfortunately, few researchers have examined the question of what data is best expressed in what way. Lacking a theory of multisensory perception and processing of information, the critical issue is determining what data "best" maps onto what sensory input channel. Consider the problem of hydrocarbon reservoirs. Most reservoir engineers would agree that the number of variables required to characterize a reservoir is large (perhaps 30 to 50). If you depend only on visual displays limited to seven variables at a time, then it might require as many as seven such displays to cover the full range of necessary variables. You then must either mentally integrate across those seven displays or go through a process of variable selection and redisplay to achieve a specific goal.