Engaging in a Conversation with Synthetic Agents along the Virtuality Continuum

Summary form only given. Most work so far has concentrated on the design and implementation of conversational agents at the two extremes of the virtuality continuum. In this article, the author reports on a new generation of synthetic characters that are no longer bound to a flat screen, but able to enter a physical world and to engage in a conversation with a human user. Users and characters do not inhabit separated spaces, but share an informational and physical reality that is augmented by digital objects. As a consequence, communication has to take into account both the physical and the digital context. New forms of deixis are enabled by the manipulation of objects and movements of characters in the physical space. Further challenges arise from the realization of so-called traversable interfaces that allow human and synthetic agents to cross the border from the digital to the real world and vice versa