Representation of Speaking Mouth Image for Visual Speech Synthesis System

The paper proposes a kind of joint feature with the geometric parameters and the color moment to represent the speaking-mouth image frames for the image-based visual speech synthesis system. The experiment results show the proposed joint feature can effectively provide the basis for classify the speaking-mouth images to distinguish the different speaking state according to the lip shape and the tooth visibility. In order to derive the geometric feature, a kind of step-by-step point feature localization is proposed here is introduced to get the feature points. Comparing the localization results by this automatic localization algorithm with that by hand, the paper proves the localization result can meet the need of geometric feature representation.