On the combination of weight-decay and input selection methods

We present the results of a research on the combination of weight-decay and input selection methods based on the analysis of a trained multilayer feedforward network. This combination has been proposed and suggested by some other authors. The influence of weight-decay in seventeen different input selection methods is empirically analyzes with a total of eight classification problems. We show that the performance variation by introducing weight-decay strongly depends on the particular input selection method. The use of weight-decay can even deteriorate the efficiency of a method. Furthermore, it seems that weight-decay improves the performance of the worst input selection methods and deteriorate the performance of the best ones. In that sense, it diminishes the performance differences among different methods. We conclude that the combination of weight-decay and this type of input selection methods should be avoided