Expressive semantics for automatic annotation and retrieval of video streams

For the practical use of video retrieval systems, access must be provided so as to bridge the semantic gap between the system and users. Semiotics, which is concerned with the production of sense and of the way in which it is received by humans appears as the formal background for this goal. According to semiotics, semantics can be extracted at different levels of signification through a suitable set of rules which combine visual and auditory signs. An intermediate semantic level, which has to do with the combination of low-level signs, such as color, motion, and shape and their changes through time, accounts for the video expressiveness and the emotions provoked. It can be obtained automatically and is often useful for retrieval by content and similarity-based classification of video streams. Applications to retrieval of commercials based on expressive semantics are discussed