Encoder assisted video frame rate up-conversion

With the use of motion compensation techniques, video frame rate up-conversion (FRC) achieves sharp and continuous motion representation and hence improves visual quality. However, motion estimation errors create artifacts that degrade the performance of FRC methods. In this paper, we propose to provide encoder assistance to improve the performance of FRC post-processing at the decoder. In this approach, encoder generates and codes any side information that could assist the decoder during FRC. The decoder uses this side information to enhance the quality of the frames that are synthesized by FRC. The proposed algorithm improves the visual quality significantly by reducing artifacts especially in occluded regions. Also, for some video contents, frames that are synthesized by encoder assisted FRC have higher average PSNR when compared to H.264 encoded B-frames at the same bitrate.