Image processing and coding for an immersive teleconferencing system

It is generally agreed that video conferencing today is limited in its support of natural human-centered communication. Body postures and subtle movement, gaze direction, room acoustics, and joint object/data interactions are often misrepresented, lacking, or wholly absent within these systems. To obviate these shortcomings, a so-called immersive teleconferencing system is being developed at HHI within the German ITI project and in cooperation with the IST project VIRTUE. Following the principle of a shared table, it subtends a class of teleconferencing systems enabling conferees located in different geographical places to meet around a virtual table, appearing at each station in such a way as to create a convincing impression of presence. This system enables the participants to make use of rich communication modalities as similar as possible to those used in a face-to-face meeting (e.g., gestures, eye contact, realistic images, correct sound direction, etc.). The author presents the following topics: state-of-the-art in video conferencing; the shared table approach; 3D image analysis and synthesis; image and disparity coding; architecture of the conference table; conclusions and outlook.