Photo classification by integrating image content and camera metadata

Despite years of research, semantic classification of unconstrained photos is still an open problem. Existing systems have only used features derived from the image content. However, Exif metadata recorded by the camera provides cues independent of the scene content that can be exploited to improve classification accuracy. Using the problem of indoor-outdoor classification as an example, analysis of metadata statistics for each class revealed that exposure time, flash use, and subject distance are salient cues. We use a Bayesian network to integrate heterogeneous (content-based and metadata) cues in a robust fashion. Based on extensive experimental results, we make two observations: (1) adding metadata to content-based cues gives highest accuracies; and (2) metadata cues alone can outperform content-based cues alone for certain applications, leading to a system with high performance, yet requiring very little computational overhead. The benefit of incorporating metadata cues can be expected to generalize to other scene classification problems.