Music and animation tool kit: Modules for computer multimedia composition

The “Music and Animation Tool Kit” is a group of computer program modules designed to be used in abstract multimedia composition. The user has direct control over the visual realm, can create patterns and textures, and can move objects freely about the screen. All visual events are translated into music. Often the same code that creates the animation also generates the music. The user therefore creates music by working through the visual domain. Two methods of translating visible movement to sound are used in these modules, coordinate mapping and perceptual mapping. Coordinate mapping is described in detail in the first part of this paper; the second part deals with perceptual mapping. In coordinate mapping, the vertices of a three-dimensional geometric shape (the points that define a form or the points where lines intersect) are assigned to notes. These notes change as the object moves. Perceptual mapping is based on principles of Gestalt psychology. In this type of mapping what is perceived visually is translated to music: it is not a literal mapping of coordinate points.