Pairwise similarities for scene segmentation combining color and depth data

The advent of cheap consumer level depth-aware cameras and the steady advances with dense stereo algorithms urge the exploitation of combined photometric and geometric information to attain a more robust scene understanding. To this end, segmentation is a fundamental task, since it can be used to feed with meaningfully grouped data the following steps in a more complex pipeline. Color segmentation has been explored thoroughly in the image processing literature, as much as geometric-based clustering has been widely adopted with 3D data. We introduce a novel approach that mixes both features to overcome the ambiguity that arises when using only one kind of information. This idea has already appeared in recent techniques, however they often work by combining color and depth data in a common Euclidean space. By contrast, we avoid any embedding by virtue of a game-theoretic clustering schema that leverages on specially crafted pairwise similarities.