Gaze correction in video communication with single camera

In face-to-face video communication, we commonly use only a single camera placed on top of a monitor screen. This general configuration gives poor eye contact problem to decrease the feeling of natural conversation since the user stares at the monitor screen rather than directly staring at the camera lens. In this paper we present a new approach for natural feeling in video communication using image-based modeling and rendering techniques. Our facial modeling approach has two components. The first component is to estimate the eye position from an input image to find the gaze-correction angle and generate the basic model to represent the user´s facial shape. The second component is a model-based shape approximation from motion which generates the user´s simple facial shape to be used in rendering. To render a good eye-contact image, we propose 3D mesh warping technique, a method to rotate input image with the correction angle and the facial model. Our approach is effective and convenient since it uses the characteristic information about a facial scene. Preliminary experimental results with real facial image shows the enhanced naturalness which the face-to-face video communication has to offer.