Searching for People through Textual and Visual Attributes

Searching for people through their personal traits has been largely required for several areas and, consequently, has become the center of attention in the scientific community. Locating a suspect or finding missing people in a public space are some of the practical applications which take advantage of research conducted in this topic. In this paper, we propose the use of describable visual attributes (e.g., male, wear glasses, has beard), as labels that can be assigned to an image to describe its appearance. The approach is based on visual dictionaries to generate an intermediate representation for the face images. We train binary classifiers for the attributes which give to each image a score used to obtain its ranking. However, there are some attributes that have no immediate antagonistic (e.g., asian people). Then, we evaluate unary classifiers for such attributes. The method is easily extensible to new attributes. For queries consisting of more than one attribute, we use two approaches of the state-of-the-art to combine the rankings: product of probabilities and rank aggregation. Experimental results show that incorporating visual dictionaries improves the accuracy for some attributes. Furthermore, for many attributes, rank aggregation achieves better results than traditional methods of rank fusion. The proposed solution might be of interest in a forensic scenario for searching suspects in a database by means of textual descriptions provided by a victim.