Mixed reality simulation for mobile robots

Mobile robots are increasingly entering the real and complex world of humans in ways that necessitate a high degree of interaction and cooperation between human and robot. Complex simulation models, expensive hardware setup, and a highly controlled environment are often required during various stages of robot development. There is a need for robot developers to have a more flexible approach for conducting experiments and to obtain a better understanding of how robots perceive the world. Mixed Reality (MR) presents a world where real and virtual elements co-exist. By merging the real and the virtual in the creation of an MR simulation environment, more insight into the robot behaviour can be gained, e.g. internal robot information can be visualised, and cheaper and safer testing scenarios can be created by making interactions between physical and virtual objects possible. Robot developers are free to introduce virtual objects in an MR simulation environment for evaluating their systems and obtain a coherent display of visual feedback and realistic simulation results. We illustrate our ideas using an MR simulation tool constructed based on the 3D robot simulator Gazebo.