A two-stage video object segmentation using motion and color information

In this paper we propose a system to provide automatic location of objects (object segmentation) in video sequences using motion and color statistics at quasi real time performance. The proposed algorithm uses both color and motion information to generate two independent models of the scene: color model and motion model. Both models are combined into a cost function that encodes the likelihood of a pixel to either belonging to the object or not. The algorithm works in a two-stage process: (1) Motion model segmentation and (2) Color model contribution. In case the intermediate results obtained at the end of the first stage (motion model segmentation) are consistent in comparison to previous frame results, stage two (color model) is skipped, allowing us to achieve 15 fps at HD resolutions, while maintaining segmentation quality. We also show an application of video object segmentation by applying our algorithm to create panoramic video summaries.