Lip parameters extraction based on projection of raw images onto reference shapes

This paper presents a method for the extraction of articulatory parameters from direct processing of raw images of the lips. After an overview of speechreading and the existing lipreading systems, a set of 23 reference lip shapes phonetically labelled, called visemes, are presented. Our system architecture can be seen as made of three independent parts. First, a new greyscale mouth image is centred and down sampled. Second, this image is projected onto a basis of images computed from the visemes. Then, a multivariate interpolation based on the visemes is applied to predict articulatory parameters from the projection coefficients