Lip contours detection and tracking with multi features

Lip contours detection and tracking has been studied extensively because it can significantly improve the performance of the automatic speech recognition and face recognition systems. A major challenge is to find a robust and accurate method for detecting and tracking lip contours. In this paper, we propose and evaluate novel method for lip detection and tracking, which is based on the concept of statistic shape models (e.g. ASM, AAM, etc) and optimization of multi features. Since, a single feature-based alignment method (e.g. normal profile ASM, Gabor ASM, snakes, etc) presents good performance only in particular conditions but gets stuck in local minima for noisy conditions. To enhance the convergence, we propose to use three features: normal profile, grey level patches and Gabor wavelets in alignment method and combine them by using a voting approach. The ASM is not able to take into account temporal information from previous frames therefore the lip contours are tracked by replacing the standard ASM with a hybrid active shape model (MF-HASM) which is capable to take advantage of the temporal information.