The face as an interface: the new paradigm for HCI

Usability should be paramount in the development of any multimodal interface. Unfortunately, the software industry market their product upgrades based on the number of additional features rather than attempting to improve the user interaction. This tends to produce packages so complex that many of the facilities remain unknown to the user and invariably unused. Instead, by using a synthetic human face or cartoon-style characters, we attempt to make the interface more transparent and begin to readdress the complexity versus usability balance. This research continues the development of the MARTI project with enhancement of facial modelling. The study has considered key work in the field of speechreading and lip reading, and has extended the domain to develop a novel conversational, American English, viseme set. Furthermore, the work considers the application of psychological ideas to facial modelling and animation, in order to create highly believable and life-like facial synthesis. New levels of visual accuracy have been achieved for both human and cartoon character animation, to attain the highest performance to date for automated, speech to face, teeth, tongue, lips, and jaw articulation