Motion-based segmentation by principal singular vector (PSV) clustering method

Motion-based segmentation has attracted a lot of attention. The task of identifying independent objects is called segmentation. Motion-based segmentation has a broad video application domain. An approach based on principal singular vectors (PSVs) of the image measurement matrix was proposed for separating independent moving objects in Kung and Yun-Ting Lin (1995). After applying SVD (singular value decomposition), feature blocks with different object-based motions tend to form separate clusters on the PSV space. Therefore, a frame can be divided into regions each with consistent motion. Our approach offers several additional features: (1) a multi-candidate feature tracker is adopted. (2) Multiple frames are utilized to facilitate motion-based separation. (3) We would like to achieve not only accurate motion estimation, but also the object regions should retain some neighborhood property (to save the bits for the coding boundary). For this, a neighborhood sensitivity parameter δ is introduced. One application of motion-based segmentation is low-bit-rate video compression. In very low bit-rate video coding, only motion vectors of finite regions and the region boundary (coded in prediction error) need to be transmitted. Yet simulations yield quite respectable compensated frames