Similarity learning in nearest neighbor, positive semi-definitiveness and RELIEF algorithm

In this paper, we develop a similarity learning version of RELIEF algorithm, called RBS-PSD (for RELIEF-Based Similarity learning) where the learned similarity matrix is projected onto the set of positive, semi-definite matrices. Unfortunately, this algorithm does not perform very well in practice since it does not try to optimize the leave-one-out error or the 0-1 loss. This motivated us to develop its stricter version, called sRBS-PSD, which aims at reducing a cost function closer to the 0-1 loss. In the case of sRBS-PSD also, the similarity matrix is projected onto the set of positive, semi-definite matrices. Experiments reveal that this projection improves the performance for all of these algorithms. Furthermore, it has been shown that sRBS-PSD outperforms its counterparts for majority of the data sets.