Optimal In-Place Learning and the Lobe Component Analysis

It is difficult to map many existing learning algorithms onto biological networks because the former require a separate learning network. The computational basis of biological cortical learning is still poorly understood. This paper rigorously introduces a concept called in-place learning. With in-place learning, every networked neuron in-place is responsible for the learning of its signal processing characteristics (e.g., efficacies of synapses) within its connected network environment. There is no need for a separate learning network. With this in-place hypothesis, consequently, each neuron does not have extra space to compute and store the second and higher order statistics (e.g., correlations) of its input fibers. This work first provides a classification of learning algorithms. Then, it shows that the two well-known in-place biological mechanisms, the Hebbian rule and lateral inhibition, are sufficient to develop orientation selective cells, similar to those found in VI, from inputs of natural images. Many other cells that have not been fully understood have emerged as well. The presented computational study of these two in-place learning mechanisms leads to a new concept- these cells correspond to what are called lobe components, which are high concentrations in the probability of the neuronal input space. Further analysis explains how every neuron can learn efficiently (i.e., near-optimal efficiency) by scheduling its plasticity while interacting with other connected neurons. A simple, in-place (Type-5) learning algorithm is presented. The experimental results showed that this simple biologically inspired algorithm is superior to some well-known state-of-the-art ICA algorithms, thanks to its near-optimal efficiency.