DocumentCode :
3748460
Title :
Listening with Your Eyes: Towards a Practical Visual Speech Recognition System Using Deep Boltzmann Machines
Author :
Chao Sui;Mohammed Bennamoun;Roberto Togneri
Author_Institution :
Sch. of Comput. Sci. &
fYear :
2015
Firstpage :
154
Lastpage :
162
Abstract :
This paper presents a novel feature learning method for visual speech recognition using Deep Boltzmann Machines (DBM). Unlike all existing visual feature extraction techniques which solely extracts features from video sequences, our method is able to explore both acoustic information and visual information to learn a better visual feature representation in the training stage. During the test stage, instead of using both audio and visual signals, only the videos are used for generating the missing audio feature, and both the given visual and given audio features are used to obtain a joint representation. We carried out our experiments on a large scale audio-visual data corpus, and experimental results show that our proposed techniques outperforms the performance of the hadncrafted features and features learned by other commonly used deep learning techniques.
Keywords :
"Visualization","Speech recognition","Feature extraction","Hidden Markov models","Acoustics","Noise measurement","Speech"
Publisher :
ieee
Conference_Titel :
Computer Vision (ICCV), 2015 IEEE International Conference on
Electronic_ISBN :
2380-7504
Type :
conf
DOI :
10.1109/ICCV.2015.26
Filename :
7410383
Link To Document :
بازگشت