Automated modeling and robotic grasping of unknown three-dimensional objects

This paper describes the development of a novel vision-based modeling and grasping system for three-dimensional (3D) objects whose shape and location are unknown a priori. Our approach integrates online computer vision-based 3D object modeling with online 3D grasp planning and execution. A single wrist-mounted video camera is moved around the stationary object to obtain images from multiple viewpoints. Object silhouettes are extracted from these images and used to form a 3D solid model of the object. To refine the model, the object´s top surface is modeled by scanning with a wrist-mounted line laser while recording images. The laser line in each image is used to form a 3D surface model that is combined with the silhouette result. The grasp planning algorithm is designed for the parallel-jaw grippers that are commonly used in industry. The algorithm analyses the solid model, generates a robust force closure grasp, and outputs the required gripper position and orientation for grasping the object. The robot then automatically picks up the object. Experiments are performed with two real-world 3D objects, a metal bracket and a hex nut. The shape, position and orientation of the objects are not known by the system a priori. The time required to compute an object model and plan a grasp was less than 4 s for each object. The experimental results demonstrate that the automated grasping system can obtain suitable models and generate successful grasps, even when the objects are not lying parallel to the supporting table.