Optimal structure from motion: local ambiguities and global estimates

We present an analysis of SFM from the point of view of noise. This analysis results in an algorithm that is provably convergent and provably optimal with respect to a chosen norm. In particular, we cast SFM as a nonlinear optimization problem and define a bilinear projection iteration that converges to fixed points of a certain cost-function. We then show that such fixed points are “fundamental”, i.e. intrinsic to the problem of SFM and not an artifact introduced by our algorithm. We classify and characterize geometrically local extrema, and we argue that they correspond to phenomena observed in visual psychophysics. Finally, we show under what conditions it is possible-given convergence to a local extremum-to “jump” to the valley containing the optimum; this leads us to suggest a representation of the scene which is invariant with respect to such local extrema