Audio-Visual Automatic Speech Recognition for Connected Digits

Audio-visual automatic speech recognition (ASR) is a hotspot in field of human-computer interaction (HCI). This paper implemented an audio-visual ASR for Chinese connected digits and addressed on the method of speech segmentation. A novel speech segmentation approach combining Otsupsilas method with traditional short-time energy and zero-crossing rate (ZCR) based method was proposed. The experimental results showed its efficiency compared with traditional method. Discrete cosine transform (DCT) coefficients and Mel frequency cepstral coefficients (MFCC) were then used as the visual/audio features respectively. After the recognition tasks for speaker-independent ASR were carried out, performances of audio-visual ASR and audio-only ASR under different noisy conditions were compared.