Motion Segmentation by EM Clustering of Good Features

We present a new algorithm that does motion segmentation by tracking small textured patches and then clustering them using EM. A small patch has the advantage that its motion is well modeled by uniform flow and runs a lower risk of boundary inclusion. Inherently, a small patch has less data so it is more susceptible to noise and it is not well suited to fit locally higher order flow models. To overcome these difficulties, we introduce a motion coherence detector to select only the best features and an efficient statistical technique to compute segment-wise affine flow from the EM clustering parameters. We incorporate a residual noise model without any statistical independence assumption and an efficient χ^2 test for the noise model to obtain dense segmentation. Computational efficiency is striven for within a rigorous mathematical framework. Experiments with real image sequences show good segments under a variety of conditions.