A visual-inertial fusion based tracking system for mobile augmented reality

Recently, smart phones are usually equipped with cameras and motion sensors. This means that mobile phones have become good platforms to deploy augmented reality (AR) applications. We propose a new tracking system for AR which uses inertial measurement units (gyroscope, accelerometer and magnetic field sensor) combined with a camera to automatically detect and track a dominant plane in the scene without prior knowledge about the environment. Meanwhile, it also estimates the pose of the device, so the rendered virtual objects can be correctly overplayed on the real view. While tracking the plane and computing the camera pose with respect to the scene, the solution also needs to construct and update the map of the scene, which consists of keyframes and keypoints in our scenario. Finally, we implement our system and test its performance on Android smart phones. Our experimental results show that the system can achieve real-time and drift-free performance and the tracking is quite accurate and robust.