Vision based object identification and tracking for mobile robot visual servo control

A key problem of an Image Based Visual Servo (IBVS) system is how to identify and track objects in a series of images. In this paper, a scale-invariant image feature detector and descriptor, which is called the Scale-Invariant Feature Transform (SIFT), is utilized to achieve robust object tracking in terms of rotation, scaling and changes of illumination. To the best of our knowledge, this paper represents the first work to apply the SIFT algorithm to visual servoing for robust mobile robot tracking. First, a SIFT method is used to generate the feature points of an object template and a series of images are acquired while the robot is moving. Second, a feature matching method is applied to match the features between the template and the images. Finally, based on the locations of the matched feature points, the location of the object is approximated in the images of camera views. This algorithm of object identification and tracking is applied in an Image-Based Visual Servo (IBVS) system for providing the location of the object in the feedback loop. In particular, the IBVS controller determines the desired wheel speeds ω_1 and ω_2 of a wheeled mobile robot, and accordingly commands the low-level controller of the robot. Then the IBVS controller drives the robot toward a target object until the location of the object reaches the desired location in the image. The IBVS system is implemented and tested in a mobile robot with an on-board camera, in our laboratory. The results are used to demonstrate satisfactory performance of the object identification and tracking algorithm. Furthermore, a MATLAB simulation is used to confirm the stability and convergence of the IBVS controller.