Real-time SLAM for static multi-objects learning and tracking applied to augmented reality applications

This paper presents a new approach for multi-objects tracking from a video camera moving in an unknown environment. The tracking involves static objects of different known shapes, whose poses and sizes are determined online. For augmented reality applications, objects must be precisely tracked even if they are far from the camera or if they are hidden. Camera poses are computed using simultaneous localization and mapping (SLAM) based on bundle adjustment process to optimize problem parameters. We propose to include in an incremental bundle adjustment the parameters of the observed objects as well as the camera poses and 3D points. We show, through the example of 3D models of basics shapes (planes, parallelepipeds, cylinders and spheres) coarsely initialized online using a manual selection, that the joint optimization of parameters constrains the 3D points to approach the objects, and also constrains the objects to fit the 3D points. Moreover, we developed a generic and optimized library to solve this modified bundle adjustment and demonstrate the high performance of our solution compared to the state of the art alternative. Augmented reality experiments in realtime demonstrate the accuracy and the robustness of our method.