Iterative Relief

Feature weighting algorithms assign weights to features according to their relevance to a particular task. Unfortunately, the best-known feature weighting algorithm, ReliefF, is biased. It decreases the relevance of some features and increases the relevance of others when irrelevant attributes are added to the data set. This paper presents an improved version of the algorithm, Iterative Relief, and shows on synthetic data that it removes the bias found in ReliefF. This paper also shows that Iterative Relief outperforms ReliefF on the task of cat and dog discrimination, using real images.