Visual recognition using mappings that replicate margins

We consider the problem of learning to map between two vector spaces given pairs of matching vectors, one from each space. This problem naturally arises in numerous vision problems, for example, when mapping between the images of two cameras, or when the annotations of each image is multidimensional. We focus on the common asymmetric case, where one vector space X is more informative than the other Y, and find a transformation from Y to X. We present a new optimization problem that aims to replicate in the transformed Y the margins that dominate the structure of X. This optimization problem is convex, and efficient algorithms are presented. Links to various existing methods such as CCA and SVM are drawn, and the effectiveness of the method is demonstrated in several visual domains.