An approach to evaluate the performance of a user affective agent

When developing agents aimed at interacting with human users, one should consider not only the specific goal that the agent was created for, but also the quality of the interaction that it offers. To achieve a specific task in a specific area, the agent needs reasoning and learning capabilities, while for offering a good, pleasant interaction, an agent has to show believable behavior and social abilities. Eventually, the performance of the agent must be evaluated. Usually this is done by collecting feedback from participants interacting with the agent, which is highly subjective. Of course, the entire experience is a subjective one, but objective measures should also exist. The goal of this paper is to define metrics for evaluating an affective agent when it performs in a human-agent interaction setting and to show how these metrics can be used on a virtual museum scenario.