Model-based versus knowledge-guided representation of non-rigid objects: a case study

Two different approaches to the detection and representation of the mouth in real video images of the human face are investigated. The model-based approach presented is based on a technique known as “deformable templates”, and tries to approximate the contours of the lips with a model consisting of four parabolas. An alternative to the model-based approach, referred to as the knowledge-guided approach, is proposed. The basic idea is not to try to capture all of the a priori knowledge about an object in a single global model that is adapted to the image, but rather to utilize the a priori knowledge in a step-by-step way, in order to refine rough initial hypotheses into a compact description of the object. This method may be interpreted as a gradual concentration on the relevant structures in the image. The combination of the resulting structures yields a compact description of the object. In the application, which is the basis for this investigation, the goal is to enhance speech recognition by using visual information about lip movements in addition to the acoustic signal. Only the problem of finding an accurate and robust representation of the lips in an image is addressed. Each of the methods were investigated for the same set of 15 faces. Our experiments indicate that the knowledge-guided approach performs more accurately and more robustly, than the model-based approach