LIP movements synthesis using time delay neural networks

A method exploiting the audio-visual correlation of speech in order to estimate the lip and mouth movements is presented. Its applications are in the field of aids and services for elderly people, in videotelephony, in cartoons and movie dubbing. Notice that lip movements synthesis does not imply speech recognition and that the mouth shape is not only specified by the phoneme currently uttered but it also depends on some past and future speech information. In order to take into account this temporal correlation, and considering the constraint of computational effectiveness, the Time Delay Neural Networks (TDNNs) seem to be the most appropriate analysis tool in comparison with methods like Markov Models, which are more resource consuming.