Stacking Jacobians properly in stereo visual servoing

Most visual servoing applications are concerned with geometrically modeled objects. In this paper, the problem of controlling a motion by visual servoing around an unknown object with a stereovision system is addressed. The main goal is to move the end-effector around the object in order to observe several viewpoints of the object for other tasks, e.g. inspection or grasping. The present work uses the well-known image-based visual servoing approach with a point, but the importance of the relationship between the end-effector and camera frames is clarified and emphasized. This relationship is needed for properly stacking the Jacobians or interaction matrices of each camera. A comparison with a visual servoing approach with a direct stacking of the Jacobians is presented. The centroid of a region, obtained by color segmentation, is used to move around the observed object. Experiments are developed on a PA-10 robot connected to a real time stereovision system, with two cameras mounted on the end-effector. Experimental results demonstrate the importance of a proper definition of the stacked Jacobians, to avoid undesired motions in the servoing task. Particularly, when turning around an unknown object, undesired motions on roll angle of the stereovision system can be avoided.