Infotainment devices control by eye gaze and gesture recognition fusion

This paper presents a novel concept for controlling the consumer devices such as MP3 player or other daily life appliances by fusion of eye gaze and gesture recognition methodologies. Such system is deployable for virtually controlling the consumer devices anywhere in the home and office. The usability of the system is also tested with patients at the intensive care units of the hospitals where the patients may not able to operate the consumer devices in a regular way. The proposed system consists of a video processing based embedded system, a CCD camera and situated display. Physical control options are displayed over the situated display. Selection / de-selection process of displayed control option over situated display is accomplished by analyzing the video sequence captured by the CCD camera. Eye gaze estimation and head gesture recognition algorithms analyze the video sequence and reveal the control command that has to be sent to the infotainment or other consumer devices.