A hierarchical semantic image labeling method via random forests

In this paper, we propose an effective image labeling method with a hierarchical framework consists of two layers of random forests. In the first layer, random forests is performed on superpixel basis. An initial labeling map is efficiently estimated by assigning every superpixel with a unique class label. In the second layer, structured random forests is applied to the image patches locating at superpixel boundaries to make use of the topological distribution of the object classes via local label descriptors. In our work, structured random forests not only generates local label predictions, but also provides us the reliability score of this prediction so that the predictions from two layers can be fused adaptively for a more reliable labeling result. This additional layer makes improvements especially on implausible label configurations and at positions where superpixel segmentation is not accurate enough. In our extensive experiments, the proposed method performs state-of-the-art accuracy.