Embodied and multimodal human-robot interaction between virtual and real worlds

The high cost and complexity to maintain a robot, and the impracticability for long term large scale HRI with real robots, have induced constraints in HRI studies. The purpose of this paper is to demonstrate the feasibility of our proposal to realize embodied and multimodal HRI between virtual robot and real world human in response to the above constraints. Based on our SIGVerse simulator, a 3D virtual environment is established for the virtual robot to interact with real world human via immersive interface. The realizations of embodiment and multimodal interaction are illustrated in several HRI research scenarios, where verbal action instruction and spatial information from the body gesture are simulated.