Lip reading based on cascade feature extraction and HMM

This paper proposes a method for building a real time lip reading system for Chinese characters. The Viola-Jones approach is adopted to detect the human face and lip area. By using this method, fast and exact extraction is accomplished. In the feature extraction module, an appearance based four-stage cascade method is proposed which includes the DCT-based and DWT-based image transformation, scanning of coefficients, PCA-based dimensional reduction and K-means based vector quantification. Then, the obtained features are applied as inputs to the Hidden Markov Model (HMM) for recognition. At last, the experimental results are included to confirm the effectiveness of the proposed system.