Automatic adaptation of a face model using action units for semantic coding of videophone sequences

The topic of investigation is automatic adaptation of a face model at the beginning of a videophone sequence for implementing mimic analysis by means of action units in a semantic coder. Here, not only the face model is to be adapted to match the real face, but also initial values of action units are to be determined. In the proposed algorithm, eye and mouth features are first estimated using deformable templates. Then, the face model Candide is adapted to these estimated features in three steps, namely: (1) the global adaptation; (2) the local adaptation; and (3) the mimic adaptation. For the mimic adaptation, six action units are used and their initial values are determined. The proposed adaptation algorithm differs from previous works in the following aspects: (1) there is no restriction on the rotation for the global adaptation of the face model and (2) initial values of action units are determined due to the mimic adaptation. The proposed algorithm has been experimented onto synthetic images and natural head-and-shoulder videophone sequences with a spatial resolution corresponding to CIF and a frame rate of 10 Hz. The average errors for the estimation of eye and mouth features and for the adaptation of the face model amount to 1.936 (pel) and 2.009 (pel), respectively. With this adaptation algorithm, mimic analysis for semantic coding by means of action units in the subsequent frames is realizable