Vocal and Facial Imitation of Humans Interacting with Virtual Agents

Socially-aware virtual agents may guide the design and deployment of future computing systems. This paper addresses unconscious affect recognition in the context of social interaction between agents and humans. An experiment is performed in which participants interact visually and verbally with virtual embodied agents. During the interaction, both the vocal pitch and the affective facial expressions of the agent are manipulated and the consecutive vocal and facial expressions of the participants are registered. Manual and computational analyses of the expressions reveal vocal and facial imitation as a sign of unconscious affect recognition and social engagement.