Synthesis-Based Texture Video Coding With Side Information

The addition of a new component to the traditional synthesis-based texture video coding algorithm is investigated in this paper. That is, we add the side information in the form of low-quality video to enhance the texture video synthesis performance with reducing the unpleasant mismatch between analyzed and synthesized regions. As compared with the conventional synthesis algorithm, our algorithm is more flexible since the behavior and quality of the output texture can be adjusted by the amount of the side information, which is determined by the user. To this end, we develop an area-adaptive side information selection scheme that chooses the proper amount of the side information for a given bit budget. Furthermore, we propose a texture decomposition scheme that extracts the non-synthesizable illumination component from the source video for separate coding so as to maximize the synthesis functionality. The superior performance of the proposed texture video synthesis technique is demonstrated by several coding examples.