DTCWT-based dynamic texture features for visual speech recognition

In this paper, new visual dynamic texture features based on dual tree complex wavelet transform (DTCWT) are proposed to capture important lip motion information for speech recognition. Lip texture features are extracted by DTCWT, for its approximate shift invariance and good directional selectivity. Canberra distances between lip texture features in adjacent frames are used as visual dynamic features. Experiments evaluations on Chinese corpus demonstrate that the performance of visual dynamic texture features is superior to static features in visual speech recognition. Compared to commonly used Canberra distances features, the dynamic texture features lead to about 8% improvement on word accuracy.