Scene-aware perceptual video coding

The mean-square-error (MSE) distortion criterion used in the state-of-the-art video coding standards, e.g. H.264/AVC and the High Efficiency Video Coding (HEVC) under standardization, is widely criticized for poor measurement of perceived visual quality. Existing research on perceptual video coding mainly employs low-level features of images/video, which cannot take into account the big picture people see. This paper proposes a scene-aware perceptual video coding scheme (SAPC), which accommodates human visual perception of the scene by reconstructing the scene from video and perform scene-based bits allocation. To be specific, more bits are allocated to the foreground object and its boundaries considering that people tend to pay more attention to the foreground and object boundaries are prone to blur at low bitrates for object occlusion. The structure from motion (SFM) technology is employed for scene reconstruction. Experiments taking HEVC as the benchmark show that our algorithm can give better visual quality than the original HEVC encoder at the same bitrate.