Multimedia systems and semiotics: collateral texts for video annotation

Multimedia content can be described at different levels of abstraction such as measures of video colour, texture and motion and comparatively high-level “semantic” descriptions in textual form. The textual description of moving images at different levels of abstraction can be explored in the context of moving images from a specialist domain, such as sequences of dance. The KAB system annotates video using collateral texts about dances produced by experts. The analysis of dance, by experts, is grounded in theories of aesthetics and semiotics: the potential contribution of such theories to a framework for multimedia computing is discussed