Higher order potentials with superpixel neighbourhood (HSN) for semantic image segmentation

Among the approaches for solving the semantic image segmentation problem that has proven successful is in formulating an energy minimization expressed on top of a conditional random field (CRF) over image pixels. Recently, high order potentials (cliques of size greater than 2) over superpixels have been incorporated in the CRF energy function yielding promising results. These potentials encourage pixels within the same superpixel to take the same label by penalizing inconsistent labeling within the superpixel. While some of the earlier attempts modeled higher order potentials without considering the conditional dependencies between superpixels, others modeled these dependencies at the cost of oversimplified models at higher levels. In this paper, we propose incorporating superpixel neighborhood information within the high order potential, hence modeling dependencies between superpixels without the need of oversimplifying or constraining the model. Results show that the proposed method achieves state-of-the-art results on the challenging PASCAL VOC 2007 dataset.