Generation of eye contact image using depth camera for realistic telepresence

In this paper, we present an eye contact system for realistic telepresence using a depth camera that utilizes an infrared structured light. In order to generate the eye contact image, we capture a pair of color and depth video and separate the foreground single user from the background. Since the raw depth data includes several types of noises, we perform a joint bilateral filtering method to reduce the noise. Then, we apply a discontinuity-adaptive depth filter to the depth map to reduce the disocclusion region. From the color image and the preprocessed depth map, we construct a three-dimensional model of the user at the virtual viewpoint. The entire system is implemented through GPU-based parallel programming for real-time processing. Finally, we obtain the gaze-corrected user. Experimental results show that the proposed system is efficient in realizing eye contact and provides more realistic experience.