Robust Structure from Motion of Nonrigid Objects in the Presence of Outlying and Missing Data

The paper focuses on robust 3D structure from motion of nonrigid objects from uncalibrated image sequences. A new affine factorization algorithm is first proposed to avoid the difficulty in image alignment for imperfect data, followed by a robust factorization scheme to handle outlying and missing data. The novelty and main contributions of the paper are as follows: (i) as a new addition to previous nonrigid structure from motion, the proposed factorization algorithm can perfectly handle imperfect tracking data; (ii) it is demonstrated that the image reprojection residuals are in general proportional to the error magnitude of the tracked features. Thus, the outliers can be detected directly from the residuals, which are then used to estimate the uncertainties of the image measurement; and (iii) the robust factorization scheme is proved empirically to be more efficient and more accurate than other robust algorithms. The proposed approach is validated and evaluated by extensive experiments on synthetic data and real image sequences.