Towards 3D vision for mobile and grasping robots

To adapt to its environment, a robot must be equipped with senses and processing which provide a usable representation of the environment. The real world is three dimensional and a system using 2D vision alone will have difficulties with, say, reliable obstacle avoidance. Ideally, 3D vision processing is required, and most of the processing must be scene-driven (bottom-up), in order to offer useful information in unexpected conditions. Stereopsis and structure-from-motion both offer direct 3D visual information (essentially from triangulation), once a correspondence of several images has been achieved. Stereo offers the advantages of short to medium range accuracy, and 3D information without sensor motion. Structure-from-motion, integrating 3D information coherently from multiple viewpoints, offers useful 3D information at long ranges (with large enough motion baseline), and the filling-in of detail obscured or not properly detected in a single view. More importantly, the coherency principle for motion allows for dynamic vision, which provides (with fast enough processing) a continually evolving representation for a moving robot. The paper concentrates on monocular vision from a moving camera