Multi-feature metric learning with knowledge transfer among semantics and social tagging

Previous metric learning approaches learn a unified metric for all the classes on single feature representation, thus cannot be directly transplanted to applications involving multiple features, hundreds to thousands of hierarchical structured semantics and abundant social tagging. In this paper, we propose a novel multi-task multi-feature metric learning method which models the information sharing mechanism among different learning tasks. We decompose the real world multi-class problems such as semantic categorization or automatic tagging into a set of tasks where each task corresponds to several classes with strong visual correlation. We conduct metric learning to learn a set of (hyper)category-specific metrics for all the tasks. By encouraging model sharing among tasks, more generalization power is acquired. Another advantage is the capability of simultaneous learning with semantic information and social tagging based on the multi-task learning framework, and thus they both benefit from the information provided by each other. Experiments demonstrate the advantages on applications including semantic categorization and automatic tagging compared with other popular metric learning approaches.