Adaptive network for optimal linear feature extraction

A network of highly interconnected linear neuron-like processing units and a simple, local, unsupervised rule for the modification of connection strengths between these units are proposed. After training the network on a high (m) dimensional distribution of input vectors, the lower (n) dimensional output will be a projection into the subspace of the n largest principal components (the subspace spanned by the n eigenvectors of the largest eigenvalues of the input covariance matrix) and maximize the mutual information between the input and the output in the same way as principal component analysis does. The purely local nature of the synaptic modification rule (simple Hebbian and anti-Hebbian) makes the implementation of the network easier, faster, and biologically more plausible than rules depending on error propagation.<>