Self-rectification and depth estimation of stereo video in a real-time smart camera system

This paper presents a self-rectification stereo vision system based on a real-time, low power, and wireless smart camera platform. The proposed self-rectification method is suitable for an embedded parallel stereo system, where the epipolar lines are parallel to the image scan lines. The stereo images are first aligned by applying 1D signature matching. Then the alignment is refined based on the quality of the disparity measurement. The rectification method can be applied both offline and online. The major advantage of this rectification method is that no clean background is needed during the rectification process. After the rectification, the conjugate epipolar line is collinear. The dense matching method is implemented to achieve the depth map. This depth map provides a tool for segmentation. The application runs in an SIMD video-analysis processor, IC3D, at 30 fps and handles disparity up to 37 pixels in CIF (320times240 pixels) mode.