Stereotype-based semantic expansion for image retrieval

We present a novel, stereotype-based semantic expansion approach to identify various image sets that stereotypically represent different aspects of a given keyword. Specifically, given an adjective keyword query, our method expands it to a set of noun sub-keywords, which are stereotypical examples that can be described by the given adjective (e.g., “cute” to “{infant, kitten, ...}”). We also perform a similar process for noun keywords with adjectives (e.g., “infant” to “{cute, sweet, ...}”). To perform such expansion, we use Google Books n-grams, a new corpus of 500 million books. We harvest stereotypical relationships among nouns and adjectives by utilizing useful lexical patterns such as similes on n-grams. To demonstrate benefits of our method, we have applied our method to text-based image retrieval. Our method shows a diverse set of images given tested keywords. According to a small scale user study with 12 participants, our method shows a higher recall ratio of what a user wants to find, compared to returning images only from original keywords.