Semantic-aware delivery of multimedia

This paper describes a system that is able to take arbitrary semantic metadata, and utilize it in the multimedia delivery decision-making process. Format independence is achieved using schema languages to describe the details of any given content or metadata, so that declarative mapping rules can be specified for translating from format-specific data points to format-independent concepts that are directly used by the framework. The system utilizes the criterion of "semantic-distortion", as an extension of Rate-Distortion Optimization based multimedia delivery. Several short video clips were encoded using H.264/SVC scalable video coding, and Scalable-To-Lossless (SLS) audio coding and adapted to four target bit rates. Subjective tests found a 72% preference for those clips which had been adapted so as to devote more bandwidth to the semantically important parts of the content when compared with standard objective-based bit-rate adaptation.