Gaze-tracking and Acoustic Vector Sensors Technologies for PTZ Camera Steering and Acoustic Event Detection

An innovative application of gaze-tracking and acoustic vector sensors (AVS) technologies for guidance of moving pan-tilt-zoom (PTZ) monitoring camera is presented. Gaze-tracking is used to steer and to zoom the camera to the gaze focus area. Additionally, it is combined with audio processing in two scenarios. First is called “audio slave”: directional acoustic monitoring is adjusted automatically to the camera direction. Second is called “audio master”: automatic detection of sound events directions is performed to take priority over user control and steer the camera towards sound source. An approach to gaze tracking is presented, utilizing new algorithmic methods for both image processing and PTZ camera steering. Then application of AVS for directional filtering of sound, and for detection of acoustic events direction is discussed. The implemented application is described, and user experience is reported. Finally, future work is discussed.