Adaptive mean shift-based image segmentation using multiple instance learning

Mean shift clustering tends to generate accurate segmentations of color images, but choosing the scale parameters remains a difficult problem which has a strong impact on its performance. We present an adaptive image segmentation framework that achieves a task-dependent top-down adaption of the scale parameters. The proposed method can be used under the context of a relevance feedback-based content-based image retrieval system. Standard mean shift clustering is used to generate an initial segmentation of the images in the database. After processing a query, the user gives the usual relevance feedback by labeling each of the images in the corresponding retrieval set as positive or negative, based on whether or not it contains a particular object of interest. In our approach, this feedback obtained as a by-product of user interaction with the system is then used in conjunction with multiple instance learning to induce a mapping from the object of interest to the scale parameters. The initial segmentation of the object of interest in each of the positive images in the database is then revised. This is done offline and is completely transparent to the user. Preliminary results indicate that the proposed method is capable of learning more informed segmentation parameters.