Lip feature fusion in speech synthesis system driven by visual-speech for speech impaired

Lipreading is applied to synthesize speech for the speech-impaired people. To get a higher recognition result, concatenative feature fusion with weighting coefficients is used to integrate the geometrical feature vector of lip region and the descriptors of lip contours by Discrete Cosine Transform to get a new discriminate vector. Experiments are carried out based on HMM with different states and Gaussian mixture component in a small database for speaker-dependent case. Experiment results showed that the integrated discriminate vector after feature fusion obtained the information from the Geometrical feature vector of lip region and the DCT descriptors of lip contours. With best weighting coefficients m: n=1.5:1, the recognition rate are improved by as much as 3.92% and 29.91%.