Using the expectation-maximization algorithm for depth estimation and segmentation of multi-view images

An algorithm for joint depth estimation and segmentation from multi-view images is presented. The distribution of the luminance of each image pixel is modeled as a random variable, which is approximated by a "mixture of Gaussians model". After recovering 3D motion, a reference image is segmented into a fixed number of regions, each characterized by a distinct affine depth model with three parameters. The estimated depth parameters and segmentation masks are iteratively estimated using an expectation-maximization algorithm, similar to that proposed in Sawhney et al. (1996). In addition, the proposed algorithm is extended for cases where more than two images are available.