Building facade interpretation exploiting repetition and mixed templates

Like many natural and other man-made objects, buildings contain repeating elements. The repetition is an important cue for most applications, and can be partial, approximate or both. This paper presents a robust and accurate building facade interpretation algorithm that processes a single input image and efficiently discovers and extracts the repeating elements (e.g. windows) without any prior knowledge about their shape, intensity or structure. The method is based on locally registering certain key regions in pairs and using these matches to accumulate evidence for averaged templates. These templates are determined via the graph-theoretical concept of minimum spanning tree (MST) and via mutual information (MI). Based on the templates, the repeating elements are finally extracted from the input image. Real scene examples demonstrate the ability of the proposed algorithm to capture important high-level information about the structure of a building facade, which in turn can support further processing operations, including compression, segmentation, editing and reconstruction.