Automatic detection of video synthesis related artifacts

An automatic approach for the detection of artifacts that may be introduced in video analysis/synthesis coding is presented. It is assumed that textures in a video scene can be classified into two categories: textures with unimportant subjective details and the remainder. We use this idea for video coding with a texture analyzer at the encoder and a texture synthesizer at the decoder. The analyzer identifies detail-irrelevant textures and generates side information for the synthesizer, which inserts synthetic textures at the specified locations. Our approach can be integrated into any video codec. This paper focuses on improvements of the texture synthesizer and the avoidance of potentially subjectively annoying spatial artifacts. The presented automatic detector yields an artifact identification rate of up to 89%. The former enables automatic online evaluation of the quality of synthesized frames, which allows eventual fallback on the reference video codec it is built on, for coding erroneous macroblocks.