Video Based VAD Using Adaptive Color Information

This paper presents a new approach for voice activity detection (VAD) in videoconferencing applications based only on video information. In the proposed approach, a face detection scheme is applied to locate the face of the participant, and anthropometric measures are used to identify one patch below each eye, as well as a neighborhood around the mouth. The patches below the eyes are used to train a skin color model suited for that specific person, which is then used to identify non-skin pixels in the mouth neighborhood. The number of non-skin pixels is taken as an estimate of the mouth openness, and its evolution across time is explored for VAD.