Frequency sensitive Hebbian learning

A new learning algorithm is proposed for the training of single layer linear networks. The network studied has an input layer of N units and an output layer of M units. The input and output layer are fully connected via an M×N weight matrix. It is well known that such a network of linear processing units will generate M principal components of the input distribution when it is trained by Hebbian type learning algorithms, such as general Hebbian algorithm (GHA). It is also known that the same network structure of winner-take-all (WTA) units will produce M cluster centres of the input space when it is trained by gradient based competitive learning, such as Kohonen learning. The new algorithm also uses a Hebbian type learning mechanism, but unlike the previous algorithms such as GHA, it simultaneously classifies the input distribution into M subclasses and extracts the principal component of each subclass distribution. To achieve robust performance, a frequency sensitive competitive learning mechanism is incorporated into the process, hence the new algorithm is called frequency sensitive Hebbian learning (FSHL). We have applied the new algorithm to image data compression applications and simulation results are presented which indicate that the new FSHL will consistently outperform the optimal Karhunen-Loeve transform (KLT) and is competitive to Kohonen networks