Robust features tracking for robotic applications: towards 2 1/2 D visual servoing with natural images

This paper deals with the robust tracking of features extracted from a sequence of images taken with an uncalibrated camera mounted on a mobile robot. Unlike most vision systems, the 3D structure of the observed objects is completely unknown. Thus, position-based visual servoing cannot be used. Similarly, one must be careful when using image-based visual servoing since the depths of the features are unknown. On the other hand, 2 1/2 D visual servoing can easily deal with unknown environments since it is only based on projective reconstruction. In order to obtain a good projective reconstruction for a safe vision-based control, we propose a multi-scale real-time approach to extract robust features. Experiments show that our algorithm can be used in robotics applications when images are noisy and uncontrolled perturbations can break the continuity of the robot motion.