The multibody trifocal tensor: motion segmentation from 3 perspective views

We propose a geometric approach to 3D motion segmentation from point correspondences in three perspective views. We demonstrate that after applying a polynomial embedding to the correspondences they become related by the so-called multibody trilinear constraint and its associated multibody trifocal tensor We show how to linearly estimate the multibody trifocal tensor from point-point-point correspondences. We then show that one can estimate the epipolar lines associated with each image point from the common root of a set of univariate polynomials and the epipoles by solving a plane clustering problem in R³ using GPCA. The individual trifocal tensors are then obtained from the second order derivatives of the multibody trilinear constraint. Given epipolar lines and epipoles, or trifocal tensors, we obtain an initial clustering of the correspondences, which we use to initialize an iterative algorithm that finds an optimal estimate for the trifocal tensors and the clustering of the correspondences using Expectation Maximization. We test our algorithm on real and synthetic dynamic scenes.