Image Segmentation and Reconstruction Using Graph Cuts and Texton Mask

In this paper, we propose a novel method for recovering the background in an image. Our method can firstly identify the foreground objects, and then the foreground can be replaced by painting with the background information. Specifically, the foreground objects are detected and removed by applying image segmentation based on graph cuts. Then we can paint the holes by employing a modified texture synthesis method based on texton mask. In the first step, the user could mark some pixels as "object" or "background", called the "seeds" in the algorithm. In particular, the region penalties in our method are defined in CIELab color space, meanwhile the pairwise similarity is based on color distance in the perceptual color space. Relying on the principle of energy minimization, the segmentation process partitions an image into two sets: background and object. The topology of our segmentation is unrestricted and both "object" and "background" can contain several isolated parts. After the "object" parts are removed, based on the surrounding information and background structure, we can find the matching patches from the texton mask obtained from the source image. After that, we select the best matching patch to fill the hole and blend the boundary areas to improve the final result. The experimental results have shown that the system can reconstruct the background of an image with complex structure.