Low-latency video tracking of horizontal, vertical, and torsional eye movements as a basis for 3dof realtime motion control of a head-mounted camera

A mobile head-mounted camera system was developed that is continuously aligned with the users direction of view. The camera looks where the eyes look. The eye movements are evaluated by an eye tracking system and used as signals to drive camera actuators for pan, tilt, and roll. The pupil detection is not sufficient to track the third degree of freedom, ocular torsion around the axis of gaze. Therefore, artificial markers are applied to the sclera of the eye to allow evaluation of all three components of eye orientation. A three-dimensional geometric model of the eyeball is required to detect these markers and calculate the angle of ocular torsion. A short calibration procedure, where the user looks at given targets, is performed to adjust the parameters to the user, and for each session. The data processing is done close-to-realtime on standard PC/laptop hardware within one to three milliseconds, making a minor contribution to the total reaction time of the system (about 36 ms). A camera guided in this way mimics the natural exploration of a visual scene, based on the sensorimotor output of a biological system for the control of eye movements, which has evolved over millions of years.