Spatial and temporal up-conversion technique for depth video

This paper proposes a novel framework for up-conversion of depth video resolution both in spatial and in time domain. Time-of-flight (TOF) sensors are widely used in computer vision fields. Although TOF sensors provide depth video in real time, there are some problems in a sense that it provides a low resolution and a low frame-rate depth video. We propose a cheaper solution that enhances depth video obtained by TOF sensor by combining it with CCD camera. The proposed method provides high quality video as a cheaper solution for low resolution, and low frame-rate depth video. It is useful when depth video is used in various applications such as 3DTV, free-view TV, teleconference system. High-quality depth video can be obtained by motion compensated frame interpolation (MCFI) and extended joint bilateral upsampling (JBU). Experimental results show that depth video obtained by the proposed method has satisfactory quality.