Estimating acoustic-labial weights in connected speech recognition systems based on HMM

Describes an approach for weighting the contribution of the acoustic and visual sources of information in a bimodal connected speech recognition system. We consider that a different acoustic-labial weight is attached to each recognition unit. The values of the weighting vector are optimised in order to minimise the error rate on a learning set. Experiments are performed on a two-speakers audiovisual database, composed of connected letters, with two different acoustic-labial speech recognition systems. For both speakers and both systems, the weights optimisation allows us to increase the recognition rate of our bimodal system