Manipulator control with image-based visual servo

A visual feedback control scheme, called image-based visual servo, is proposed for manipulators with cameras on their hands. To accomplish specific tasks in unstructured environments, it is essential to have capabilities of recognizing the object position with respect to the manipulator hand. Using the Jacobian matrix relating the camera motion to the object position change in the acquired image, a state space formulation of a visual feedback system is established. On the basis of the task defined in the image plane, the desired motion of the hand is achieved by time-variant state feedback. The image-based scheme is applied to a task of tracking a moving object by the camera (and the hand). Simulations show that the control scheme is stable even under noisy conditions, while the conventional position-based servo tends to be unstable. Experiments on a PUMA 560 show the validity and the effectiveness of the image-based visual servo