A relation between Hebbian and MSE learning

Traditionally, adaptive learning systems are classified into two distinct paradigms-supervised and unsupervised learning. Although a lot of results have been published in these two learning paradigms, the relations between them have been seldom investigated. We focus on the relationship between the two kinds of learning and show that in a linear network the supervised learning with mean square error (MSE) criterion is equivalent to the basic anti-Hebbian learning rule when the desired signal is a zero mean random noise independent of the input. At least for this case there is a simple relationship between the two apparent different learning paradigms