Latent semantic retrieval of personal photos with sparse user annotation by fused image/speech/text features

While users prefer high-level semantic photo descriptions (e.g., who, what, when, where), we wish to minimize the need to annotate photos using such descriptions by the user. We propose a latent semantic personal photo retrieval approach using fused image/speech/text features. We use low-level image features to derive relationships among sparsely annotated photos, and probabilistic latent semantic analysis (PLSA) models based on fused image/speech/text features to analyze photo ldquotopicsrdquo. We then retrieve the photos using text or speech queries of simple high-level semantic words only. In preliminary experiments, while only 10% of the photos were manually annotated, the photos could be well retrieved with very encouraging results.