Audio-visual speech recognition in a Portuguese language based application

We present in this article experimental results obtained with an automatic speech recogniser developed for a speaker dependent and continuous speech alphanumeric recognition application based on the European Portuguese language. An audio-visual speech recognition approach was followed to design and build this system. Besides the well known complementary between the acoustic and the visual information for speech recognition purposes, the visual features are obviously immune to any acoustic disturbance, thus making the system more robust in acoustically contaminated environments. The results presented clearly show that the inclusion of a video stream, using a multi-stream decoding formalism, decreases the word error rate in approximately 56%_rel over a wide range of acoustical signal-noise ratio.