Solving the problem of overfitting of the pseudo-inverse solution for classification learning

The authors investigate the pseudoinverse solution for the learning of a binary classification. They address the problem of overfitting of this solution, i.e. the fact that the generalization rate can be relatively low although the learning rate is very high. They interpret this phenomenon with respect to the standard deviation of the eigenvalues of the covariance matrix of the learned patterns. The authors propose two ways to solve this problem: the first one is linear, and the second one is a two-layer perceptron. Numerical simulations are given to illustrate these approaches.<>