Dynamic reconstruction of 3D structure and 3D motion

This paper presents a computational scheme for reconstructing 3D structure, 3D motion and complete surfaces from instantaneous image motion based on multiple frames. The scheme is motivated by psychophysical observations and designed to achieve efficient, robust and flexible behavior. The scheme consists of multiple interacting stages. At the structure-from-motion stage, relative depths and 3D velocities are estimated by minimizing the error in fit to the instantaneous 2D motion measurements and the overall deviation from rigidity. This nonlinear optimization is achieved by an efficient two-stage iterative algorithm. The temporal integration stage based on Kalman filtering effectively improves the 3D structure over multiple frames. The global motion of the observer or each constituent object can be computed at a later stage by using the estimated 3D velocities. The surface reconstruction stage reconstructs complete 3D surfaces, which can further stabilize the 3D structure recovery. The scheme can account for a number of psychophysical findings that include the perceptual demonstrations explored by Ramachandran et al. (Husain et al. 1989, Lappin et al. 1980) regarding interactions between two superimposed cylinders, and the experimental results using displays of moving points with short lifetimes [28, 55]