A lip reading method based on 3-D DCT and 3-D HMM

Lip reading aims at recognizing what human says by analyzing visual speech information, such as lip movement. This technique is used to improve the recognition rate under the noise environment. Now, more lip feature extracting methods are developed, most of which are sensitive to lip positioning. To extract the features using Hidden Markov Model can be avoided such defect. Lip movement videos have 3-D structure. Therefore, we extend Hidden Markov Models to 3-dimensional space, construct a lip movement model using it. In order to consider lip dynamic features, we extract the feature vectors using 3-D Discrete Cosine Transform. 3-D DCT and 3-D HMM based lip reading method has following advantages: It can consider dynamic features of the lip movement. It can be robust on rotation, parallel shift and variant scaling. We tested its performance on VidTIMIT database compared with Pseudo 3-D HMM based method. Our method can increase the recognition rate about 2~3% against P3-D HMM based method.