Improved video segmentation through robust statistics and MPEG-7 features

Video segmentation is an important task for a wide range of applications like content-based video coding or video retrieval. In this paper, a new spatio-temporal video segmentation framework is presented. It is based upon robust statistics, namely an M-estimator, and incorporates an MPEG-7 descriptor for consistent temporal labeling of identified textures. The algorithm is based on assumptions about the geometric modifications a given moving region undergoes with time as well as on its surface properties. Homogeneously moving segments are described using a parametric motion scheme. The latter is used to piecewise fit the optical flow field in order to extract rigid motion areas. Robust statistics are used to carefully constrain split, merge and contour refinement decisions. Experimental results show that regions detected by the proposed method are more reliable than the state-of-the-art. True region boundaries are moreover better detected.