Embodied coversational agents: A methodology for learning to express facial emotions

Embodied Conversational Agents (ECAs) constitute a special type of agents which can simulate verbal and non-verbal behaviors in order to achieve more natural interaction with humans. In this paper, we present a methodology that can be used to assist an ECA to generate and express facial emotions by properly deforming its facial characteristics. The methodology has been implemented on Greta agent and consists of two main stages. Initially, facial expressions are generated by Greta by setting random values to the Facial Animation Parameters (FAPs). Then, a Multi-Layer Perceptron Neural Network (MLPNN) is trained, based on the parameters of the facial deformation of the expressions and the corresponding expert´s annotations, to model and recognize the emotional content of new facial expressions. The expressions used for the training of the MLPNN were emotionally annotated by a human expert, who had determined for each one whether it is a neutral or an emotional expression and in case it is an emotional, had specified the emotions it conveys and their strength. In this approach, Greta can generate new facial expressions, recognize the emotional content of each one, and implement the facial expression of the desired emotional state. The evaluation study conducted showed very encouraging results.