The augmented man

“The augmented man” presents a vision of a new media technology, blending real and virtual worlds, and has been developed in the context of an artistic exhibition. The visitor meets a virtual man, who seems to have emotions and who tries to get in contact by talking and asking questions. The application consists of augmented stereo videos, which are projected on a wall projection screen. The visitor sees himself as if in a 3D mirror, but with a new virtual person walking beside him. The augmented man reacts to people by approaching them or avoiding them. This paper gives a complete overview of the exhibition and presents the artistic concepts and the technical realization. After explanations about our motivation and background ideas, the modelling and the behavioural coding of the virtual man are described. Rendering aspects, like multi-texturing and 3D augmented videos are discussed in detail. The determination of the positions of the visitors is achieved by processing the images of a camera fixed on the room´s ceiling. Combined with the stereo front-facing cameras, we show how occlusions between the real person and the virtual man can be handled. The system works in real-time on a single machine with three video input streams and two graphic outputs