Feature extraction and calibration for stereo reconstruction using non-SVP optics in a panoramic stereo-vision sensor

Omni-directional sensors are useful in obtaining a 360° field of view with a single lens camera. Omni-directional stereo imaging systems were designed in our past research using a single camera and a mirror consisting of two concentric, radially symmetric lobes. If the central camera-mirror axis is vertical, a stereo image containing imagery from all azimuth directions from, two viewpoints can be captured within one image. A vertically posed catadioptric optical system that employs a mirror with a radial profile other than parabolic or hyperbolic cannot maintain the straightness of any nonvertical lines. However other mirror profiles are desirable for resolution distribution, sensor size and manufacturability. These other mirror shapes, including spherical mirrors, are said to be non-SVP (Single View-Point). Since they lack a virtual perspective point, they pose new feature extraction challenges. A method for processing the imagery from a panoramic non-SVP catadioptric stereo sensor to reconstruct a three-dimensional model of horizontal and vertical line features is introduced. Horizontal line segments are extracted using the panoramic Hough transform and vertical line segments are recognized as straight radial lines. These segments, and closed shapes they form are matched between the two lobe views to locate them in three-dimensions. The practical accuracy obtainable with such a system was explored, and some of the issues addressed to make this sensor work in real 3D reconstructions are described. Triangulating a 3D location requires better calibration information than is required for the robust feature extraction.