Non-linear neighborhood component analysis based on constructive neural networks

In this paper, we propose a novel non-linear supervised metric learning algorithm. The algorithm combines the neighborhood component analysis method with constructive neural networks which gradually increase the network size during the training process. The network aims to maximize a stochastic variant of the leave-one-out K-nearest neighbor (KNN) score on the training set. In this way, the proposed algorithm learns a nonlinear metric for KNN classification, overcoming the limitations of traditional metric learning algorithms which are only capable of learning linear transformations. Therefore, the proposed method is more flexible and powerful in transforming data than its linear counterpart. Moreover, it can also learn a low-dimensional non-linear mapping for visualization and fast classification. We validate our method on several benchmark datasets both for metric learning and dimensionality reduction, and the results demonstrate the competitiveness of the proposed approach.