Development and evaluation of a vision algorithm for 3D reconstruction of novel objects from three camera views

When planning robotic grasping and manipulation maneuvers, knowledge of the shape and pose of the object of interest is critical information. In order for an autonomous or semi-autonomous system to operate intelligently in an unstructured environment and interact with novel objects, it must have the ability to recover this information at run time, even when no a priori information of the object is available. In this paper, we describe the development and testing of an algorithm that can reconstruct the full 3D geometry of a novel object from just three images. A variant of shape from silhouettes, the algorithm first generates a rough surface approximation in the form of a point cloud. This approximation is then refined by fitting an eleven parameter geometric surface to the points in such a manner that the surface ignores noise and perspective projection shadows. We test the algorithm in both simulation and on several real world objects. We show that the algorithm provides accurate reconstructions that can be directly used to plan grasping maneuvers. Compared to other attempts in the literature, the proposed algorithm is faster, requires fewer images, is more accurate, and degrades gracefully in the presence of bad data. A real world test case is included that shows that the algorithm still yields usable results when the form of the object is amorphous or otherwise non-geometric.