Pairwise Similarities across Images for Multiple View Rigid/Non-Rigid Segmentation and Registration

A variational approach for the background/foreground segmentation of multiple views of the same scene is presented. The main novelty is the introduction of cost functions based on pairwise similarity between pixels across different images. These cost functions are minimized within a level set framework. In addition, a warping model (rigid or non-rigid) between the emerging foregrounds in the different views is imposed, thus avoiding the introduction of a specific shape term in the cost function to handle occlusions. The thin plate spline (TPS) warping is for the first time employed within the level set framework to model non- rigid deformations. The minimization of these cost functions leads to simultaneous segmentation and registration of the different views. Examples of segmentations of a variety of objects are shown and possible applications are proposed.