Multi-instance Object Segmentation with Exemplars

We address the problem of joint detection and segmentation of multiple object instances in an image, a key step towards scene understanding. Inspired by data-driven methods, we propose an exemplar-based approach to the task of multi-instance segmentation using a small set of annotated reference images. We design a novel CRF model that jointly models object appearance, shape deformation, and object occlusion at the super pixel level. To tackle the challenging MAP inference problem, we derive an alternating procedure that interleaves object segmentation and layout adaptation.