Joint image segmentation and motion estimation for low bit rate video coding

This paper proposes a joint image segmentation and motion estimation technique for very low bit rate video coding. In region-based coding, a segmentation algorithm is indispensable. Motivated by the observations of the conventional algorithm, we propose a jointly optimized image segmentation and motion estimation algorithm. In our algorithm, the input image is split into a quadtree structure, and the split regions are merged by a greedy merging algorithm (GMA). In each step of GMA, two regions which have the minimum cost are merged. Since we have chosen the cost as the increase of displaced frame difference (DFD) energy, the image is segmented into a set of regions with minimum DFD energy. By computer simulation, we show the segmentation results have much lower DFD energy compared to the block-based motion estimation