Speech to facial animation conversion for deaf customers

Author

Takacs, Gyorgy ; Tihanyi, Attila ; Bardi, Tamas ; Feldhoffer, Gergely ; Srancsik, Balint

Author_Institution

Fac. of Inf. Technol., Peter Pazmany Catholic Univ., Budapest, Hungary

fYear

2006

fDate

4-8 Sept. 2006

Firstpage

1

Lastpage

5

Abstract

A speech to facial animation direct conversion system was developed as a communication aid for deaf people. Utilizing the preliminary test results a specific database was constructed from audio and visual records of professional lip-speakers. The standardized MPEG-4 system was used to animate the speaking face model. The trained neural net is able to calculate the principal component weights of feature points from the speech frames. The control coordinates have been calculated from PC weights. The whole system can be implemented in standard mobile phones. Deaf persons were able correctly recognize about 50% of words from limited sets in the final test based on our facial animation model.

Keywords

computer animation; handicapped aids; mobile computing; neural nets; principal component analysis; speech processing; MPEG-4 system; PC weights; communication aid; deaf customer; facial animation model; mobile phone; neural net; principal component weights; speaking face model; speech frame; speech to facial animation conversion; Databases; Face; Facial animation; Speech; Videos; Visualization;

fLanguage

English

Publisher

ieee

Conference_Titel

Signal Processing Conference, 2006 14th European

Conference_Location

Florence

ISSN

2219-5491

Type

conf

Filename

7071139