Information integration for accurate foreground segmentation in complex scenes

The authors propose a hybrid framework that combines frame difference and background subtraction to integrate complementary sources of information for monocular video segmentation. This framework is modelled as an optimisation process of an energy function, which is established on a Markov random field (MRF) and optimised by Gibbs sampling. It provides a way to exploit different kinds of information obtained from frame difference and background subtraction. Central to the proposed method are two facts - that shape prior can be flexibly obtained from frame difference, and shadow removal can be integrated into the framework with a background texture model. The experiments show that this approach reliably and accurately performs on sequences that include different scenarios (indoors, outdoors) and also addresses several canonical segmentation problems, such as camouflage, foreground aperture and so forth.