Motion-tolerance contextual visual saliency preserving for video retargeting

In this paper, in order to detect the focus of interest with motion-tolerance we propose a novel approach for modeling visual dynamics based on spatiotemporal slice (STS), which can provide rich visual patterns along a large temporal scale. Patch-based visual patterns are first computed to generate a codebook in the automatic specified spatiotemporal extent that is determined by contextual information in STS. The codebook is then used to compute its associated response in video frames and eventually an importance map for each frame covering the focus of interest can be obtained. To preserve the visual continuity an optimal video deformation constrained by the trajectories in STS is determined based on a multi-cue approach. Experimental results show that the proposed approach of video retargeting based on STS can effectively generate retargeted videos while maintain their isotropic manipulation and the continuous dynamics of visual perception.