Visual feature extraction for isolated word visual only speech recognition of Vietnamese

This paper presents our research on visual feature extraction with some special treatment for dealing with Vietnamese language. The effect of linear discriminant analysis (LDA) when training with different sets of basic class will be examined. For improving the visual features, we proposed two types of visual front end for automatic lip-reading: (a) 1-Stage LDA visual front end; and (b) hierarchical LDA (HLDA) visual front end. We also compare four different types of visual feature on an isolated word visual only speech recognition of Vietnamese task using our recorded audio-visual speech database. Experiments on our database show that the proposed visual front end improves up to 8% of recognition accuracy and the HLDA visual front end outperform the other.