Robust Relocalization and Its Evaluation for Online Environment Map Construction

The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation relocalization, using virtual keyframes for online environment map construction. In the case of tracking loss, incoming camera frames are matched against known-orientation keyframes to re-estimate camera orientation. Instead of solely using real keyframes from incoming video, the proposed approach employs virtual keyframes which are distributed strategically within completed portions of an environment map. To improve tracking speed, we introduce a new variant of our system which carries out relocalization only when tracking fails and uses inexpensive image-patch descriptors. We compare different system variants using three evaluation methods to show that the proposed system is useful in a practical sense. To improve relocalization robustness against lighting changes in indoor and outdoor environments, we propose a new approach based on illumination normalization and saturated area removal. We examine the performance of our solution over several indoor and outdoor video sequences, evaluating relocalization rates based on ground truth from a pan-tilt unit.