A superimposed self-character mediated video chat system with the function of face-to-face projection based on talker´s face direction

We previously proposed an embodied video communication system called “E-VChat,” in which a CG avatar is superimposed on the other talker´s video image in a face-to-face scene, so that they can better comprehend their mutual interaction during remote communication. Remote talkers can then communicate interactively, via their own avatar, with the partner´s video image, which moves based on the talker´s motion, and which automatically generates motion based on the on-off pattern of the talker´s voice. In its practical use for video communication, images of the talkers are taken from various viewpoints around the monitor. In our previous system, however, the talker´s self-character is placed at the bottom center of the partner´s image, irrespective of the partner´s angle. In this paper, we examine the effects of a virtual face-to-face projection based on the talker´s face angles and of the selection of the talker´s avatar character in this E-VChat system for enhanced avatar-mediated communication. We demonstrate the effectiveness of a self-avatar that uses the talker´s head motion and auto-generated motion based on speech input through a communication experiment in which 12 pairs of subjects are evaluated. In addition, through a communication experiment, we confirm the effectiveness of the superimposed avatar for a face-to-face floating projection based on the partner´s face angle, as compared to bottom-center projection of the partner´s image, irrespective of the partner´s angle.