Gaze tracking for multimodal human-computer interaction

This paper discusses the problem of gaze tracking and its applications to multimodal human-computer interaction. The function of a gaze tracking system can be either passive or active. For example, a system can identify user´s message target by monitoring the user´s gaze, or the user could use his gaze to directly control an application or launch actions. We have developed a real-time gaze tracking system that estimates the 3D position and rotation (pose) of a user´s head. We demonstrate the applications of the gaze tracker to human-computer interaction by two examples. The first example shows that gaze tracker can help speech recognition systems by switching language model and grammar based on user´s gaze information. The second example illustrates the combination of the gaze tracker and a speech recognizer to view a panorama image