A noise-based stability evaluation of threshold-based feature selection techniques

This paper presents a noise-based stability performance evaluation approach for feature selection techniques. For the stability assessment, a similarity-based measure is used to quantify the degree of agreement between a filter´s output on a clean dataset and its outputs on the same dataset corrupted with different combinations of noise level and noise distribution. Experiments are conducted with 11 threshold-based feature selection techniques on six different real-world datasets. The experimental results show that some filters perform much better than others in terms of their insensitivity to noise. The results also show an interesting relationship between the size of the training data and the stability performance of a filter; the stability performance of a filter tends to improve when learning from large size datasets.