Eye-based HCI with Full Specification of Mouse and Keyboard Using Pupil Knowledge in the Gaze Estimation

Eye-based Human Computer Interaction (HCI) with full specification of mouse and keyboard by human eyes only is proposed for, in particular, disabled person and for input device of wearable computing. It utilizes pupil knowledge in gaze estimation process. Existing conventional eye-mouse, gaze-mouse does not robust against various types of users with different features of color, shape and size of eyes and also is affected by illumination changes, users\´ movement (attitude changes), etc. Using knowledge about features, the influences are eliminated. Also the proposed eye-based HCI system allows simultaneous key input of more than three keys, for instance, "Ctl+Alt+Del" for initiating task manager, left/right click and drag/drop of mouse event capabilities. Although currently commercially available screen keyboard allows simultaneous key input for two keys such as "shit+*", "Alt+*", it does not allow three keys input at once. Such these full specification of mouse events and keyboard functions are available for the proposed HCI system. Experimental results with six different nationalities of users with the different features shows effectiveness of using the knowledge for improvement of gaze estimation accuracy.