Direct estimation of motion and extended scene structure from a moving stereo rig

We investigate the relationship between the kinematics (infinitesimal motion model) of a calibrated Stereo Rig and point and line image feature measurements seen at two time instances of the rig´s motion (four images in all). In particular we are interested in the byproduct of this analysis providing a direct connection between the spatio-temporal derivatives of the images at two time instances and kinematics of the 3D motion of the Rig. We establish a fundamental result showing that 3 quadruples of point-line-line-line matches (i.e., point in the reference image and lines coincident with the corresponding points in the remaining three images) are sufficient for a unique linear solution for the kinematics of the rig. In other words, the projected instantaneous motion of “one and a half” 3D lines is sufficient for recovering the kinematics of the moving rig. In particular, spatio-temporal derivatives across 3 points are sufficient for a direct estimation of the rig´s motion. Consequently, we describe a new direct estimation method for motion estimation and 3D reconstruction from stereo image sequences obtained by a stereo rig moving through a rigid world. Correspondences (optic flow) are not required as spatio-temporal derivative are used instead. One can then use the images from both pairs combined, to compute a dense depth map. Finally, since the basic equations are linear, we combine the contribution coming from all pixels in the image using a Least Squares approach