Maximizing the margin with feedforward neural networks

Feedforward neural networks (FNNs) and support vector machines (SVMs) are two machine learning frameworks developed from very different starting points of view. In this work a new learning model for FNNs is proposed such that, in the linearly separable case, tends to obtain the same solution of that SVMs. The key idea of the model is a weighting of the sum-of-squares error function, which is inspired in the AdaBoost algorithm. The model depends on a parameter that controls the hardness of the margin, as in SVMs, so that it can be used for the nonlinearly separable case as well. In addition, it allows one to deal with multi-class and multi-label problems in a natural way (as FNNs usually do), and it is not restricted to the use of kernel functions. Finally, it is independent of the concrete algorithm used to minimize the error function. Both theoretic and experimental results are shown to confirm these ideas