Real-time segmentation of objects from video sequences with non-stationary backgrounds using spatio-temporal coherence

A real-time video segmentation algorithm, which can extract objects from video sequences even with non-stationary backgrounds, is proposed in this work. First, we segment the first frame into an object and a background interactively to build the probability density functions of colors in the object and the background. Then, for each subsequent frame, we construct a coherence strip, which is likely to contain the object contour, by exploiting spatio-temporal correlations. Finally, we perform the segmentation by minimizing an energy function composed of color, coherence, and smoothness terms. Experimental results on various test sequences show that the proposed algorithm provides accurate segmentation results in real-time, even though video sequences contain unstable camera motions.