Stereo vision and segmentation

I will describe models and algorithms for the real-time segmentation of foreground from background layers in stereo video sequences. Automatic separation of layers from color/contrast or from stereo alone is known to be error-prone. Here, color, contrast and stereo matching information are fused to infer layers accurately and efficiently. The stereo-match likelihood is then fused with a contrast-sensitive color model that is learned on the fly, and stereo disparities are obtained by dynamic programming. Our "layered graph cut" (LGC) algorithm, does not directly solve stereo. Instead the stereo match likelihood is marginalized over disparities to evaluate foreground and background hypotheses, and then fused with a contrast-sensitive color model. Segmentation is solved efficiently by graph cut optimization. In a recent development, this segmentation procedure has been used, in turn, to improve the efficiency of stereo matching, by exploiting Panum fusional bands that are well known to operate in human stereo vision.