Video Segmentation and Compression using Hierarchies of Gaussian Mixture Models

We present a new method that permits arbitrary region description in video sequence using a reduced number of bits allowing promising results in video compression. A robust unsupervised 3D (2D + t) segmentation is used to detect arbitrary regions for motion compensated video coding. The video sequence is first modeled using a Gaussian mixture model where each pixel, defined by its spatiotemporal position and its color vector is supposed to be generated by one of the mixture components. This permits to segment the video sequence into objects each one modeled by one Gaussian distribution. Grouping the mixtures in a binary tree defines a hierarchical representation of the video objects and a gradual segmentation. This segmentation is then used for region description in a motion compensated video coder. This provides a large improvement in motion bits budget. When compared to a H.264 video coder, promising results were obtained.