Multimodal topic modeling based geo-annotation for social event detection in large photo collections

Multimodal image clustering becomes an effective approach for social event detection in large photo collections. In addition to visual and textual information, geographic information can also be used to improve the detection accuracy of social events. However, not every image in a photo collection is tagged with geographic information. A topic model based approach is proposed to estimate missing geographic information in a photo which involves a supervised multimodal model to estimated the joint distribution of time, geographic, content, and textual information for a large set of photos. The photos without geographic information are annotated with a predicted geographic coordinate. We show the efficacy of the proposed approach for event detection and annotation from a large photo collection.