Geometrical Feature Extraction for Robust Speech Recognition

Visual information from lip contour has been successfully shown to improve the robustness of automatic speech recognition especially in noisy environments. In this paper, a novel method for lip reading is presented. In the method, hue information of input images is used for lip area detection. Then, a set of morphological operations is applied to detect lip contour. Polynomial fitting is designed for geometrical feature extraction. With the extracted features, hidden Markov models and Gaussian mixture models are trained to recognize speech. The experimental results demonstrated that the proposed method improved speech recognition rates in noisy environment. Another advantage of the method is its robustness to lighting variances