Feature-level Fusion for Object Segmentation using Mutual Information

We present a new feature-level image fusion technique for object segmentation based on mutual information. Using object regions roughly detected from one sensor as input, the proposed technique extracts relevant information from another to complete the segmentation. First, a contourbased feature representation is presented that implicitly captures object shape. The notion of relevance across sensor modalities is then defined using mutual information computed based on the affinity between contour features. Finally a heuristic selection scheme is proposed to identify the set of contour features having the highest mutual information with the input object regions. The approach works directly from the input image pair without relying on a training phase. Results are presented for segmenting people from background, and quantitatively evaluated.