Camera localization using mutual information-based multiplane tracking

This paper deals with dense visual tracking robust towards scene perturbations using 3D information to provide a space-time coherency. The proposed method is based on an piecewise-planar scenes visual tracking algorithm which aims to minimize an error between an observed image and a reference template by estimating the parameters of a rigid 3D transformation taking into acount the relative positions of the planes in the scene. The major drawback of this approch stems from the registration function used to perform the minimization (the sum of squared differences) as it is very poorly robust towards scene variations. In this paper, the tracking process is adapted to take into account two more complex registration functions. First, the sum of conditional variance. Since it is invariant to global illumination variations, the proposed algorithm is robust with relation to those conditions whilst keeping a low computation complexity. Then, the mutual information is considered. In that case the complexity is greater but so is the robustness towards non global illumination variations, specularities or occlusions. The proposed approaches, after being described, are tested on different scenes under varying illumination conditions to assess their respective efficiency.