Crossmodal combination among verbal, facial, and flexion expression for anthropomorphic acceptability

This paper proposes an effective communication with an agent´s appearance and reality of the acceptance level of the user´s order using verbal and nonverbal expressions. The crossmodal combination is expected to enable delicate expressions of the agent´s internal state especially when the agent´s decision of the acceptance is different from the agent´s mind, so the user becomes aware of the difficult situations. We have proposed to adopt the facial expression and the flexion of the word ending as the parameters of the agent´s internal state. The expressions are attached to the scripts of each acceptance level. The results of the expressions showed the following: 1) the effectiveness of both the facial and flexion expressions, and 2) the crossmodal combinations that generate the agent´s concealed but real feeling.