Hierarchical ranking of facial attributes

We propose a novel hierarchical structured prediction approach for ranking images of faces based on attributes. We view ranking as a bipartite graph matching problem; learning to rank under this setting can be achieved through structured prediction techniques that directly optimize the matching measures. Our key contribution is a novel model that combines structured predictors for different feature descriptors in a hierarchical fashion, enabling accurate ranking. We demonstrate our method on an important application which consists of searching for people over short intervals of time based on facial attributes. Given queries containing physical traits of a person (e.g., red hat, beard, and sunglasses), and an input database of face images, our system ranks the images in the database according to the query. Experiments show that our proposed hierarchical ranking approach poses significant enhancements in terms of accuracy over the non-hierarchical baseline.