Comparison of MLP cost functions to dodge mislabeled training data

Multilayer perceptrons (MLP) are often trained by minimizing the mean of squared errors (MSE), which is a sum of squared Euclidean norms of error vectors. Less common is to minimize the sum of Euclidean norms without squaring them. The latter approach, mean of non-squared errors (ME), bears implications from robust statistics. We carried out computational experiments to see if it would be notably better to train an MLP classifier by minimizing ME instead of MSE in the special case when training data contains class noise, i.e., when there is some mislabeling. Based on our experiments, we conclude that for small datasets containing class noise, ME could indeed be a very preferable choice, whereas for larger datasets it may not help.