Perceptual Optimization for Scalable Video Compression Based on Visual Masking Principles

This paper describes a visual optimization strategy for scalable video compression. The challenge scalable coding presents is that truncation of an embedded codestream may induce variable and highly visible distortion. To overcome the deficiencies of visually lossless coding schemes, we propose using an adaptive masking slope to model the perceptual impact of suprathreshold distortion arising from resolution and bit-rate scaling. This allows important scene structures to be better preserved. Following visual masking principles, local sensitivity to distortion is assessed within each frame. To keep the perceptual response uniform against spatiotemporal errors, we mitigate errors compounded by the motion field during temporal synthesis. Visual sensitivity weights are projected into the subband domain along motion trajectories via a process called perceptual mapping. This uses error propagation paths to capture some of the noise-shaping effects attributed to the motion-compensated transform. A key observation is that low contrast regions in the video are generally more susceptible to unmasking of quantization errors. The proposed approach raises the distortion-length slope associated with these critical regions, altering the bitstream embedding order so that visually sensitive sites may be encoded with higher fidelity. Subjective evaluation demonstrates perceptual improvement with respect to bit-rate, spatial and temporal scalability.