Semi-supervised Learning of Geospatial Objects through Multi-modal Data Integration

We investigate how overhead imagery can be integrated with non-image geographic data to learn appearance models for geographic objects with minimal user supervision. While multi-modal data integration has been successfully applied in other domains, such as multimedia analysis, significant opportunity remains for similar treatment of geographic data due to location being a simple yet powerful key for associating varying data modalities, and the growing availability of data annotated with location information either explicitly or implicitly. We present a specific instantiation of the framework in which overhead imagery is combined with gazetteers to compensate for a recognized deficiency: most gazetteers are incomplete in that the same latitude/longitude point serves as the bounding coordinates of the spatial extent of the indexed objects. We use a hierarchical object appearance model to estimate the spatial extents of these known object instances. The estimated extents can then be used to revise the gazetteers. A particularly novel contribution of our work is a semi-supervised learning regime which incorporates weakly labelled training data, in the form of incomplete gazetteer entries, to improve the learned models and thus the spatial extent estimation.