Video segmentation using vector-valued diffusion and clustering

In this paper, we propose a user-assisted video segmentation algorithm based on color information to alleviate oversegmentation problems. We perform intra-frame segmentation by image simplification, region labeling, and color clustering. In this paper, we also present a discrete three dimensional diffusion model for easy implementation. The statistical property of each labeled region is used to estimate the number of total clusters, and agglomerative hierarchical clustering is performed with the estimated number of clusters. Since the proposed clustering algorithm counts each region as a unit, it does not generate oversegmentation problems along region boundaries. For inter-frame segmentation, we employ a look-up table for foreground color clusters, track the foreground regions, and utilize those information to extract moving objects.