Lip temporal pattern analysis for automatic visual speech recognition

This paper presents a novel approach to processing temporal lip motion information for dynamic visual feature extraction in visual speech recognition. The long-time lip temporal patterns (LipTRAPs) of visual phonemes are introduced to analyze the nature of lip shape changes when uttering speech. A dynamic visual feature is also proposed based on the LipTRAPs. Visual speech recognition experiments on a connected-digits task show that the LipTRAP feature can yield significant WRR improvements than conventional delta features.