Multimodal interaction in collaborative virtual environments

Human interfaces for computer graphics systems are now evolving towards a total multi-modal approach. Information gathered using visual, audio and motion capture systems are now becoming increasingly important within user-controlled virtual environments. This paper discusses real-time interaction through the visual analysis of human face feature. The underlying approach to recognize and analyze the facial movements of a real performance is described in detail. The output of the program is directly compatible with MPEG-4 standard parameters and therefore enhances the ability to use the available data in any other MPEG-4 compatible application. The real-time facial analysis system gives the user the ability to control the graphics system by means of facial expressions. This is used primarily with real-time facial animation systems, where the synthetic actor reproduces the animator´s expression. The MPEG-4 standard mainly focuses on networking capabilities and it therefore offers interesting possibilities for teleconferencing, as the requirements for the network bandwidth are quite low