Mean-variance backpropagation: a connectionist learning algorithm with a selective attention mechanism

A modified version of the backpropagation learning algorithm called mean-variance backpropagation (MV-BP) is presented. It uses gradient descent to minimize a weighted mixture of the overall mean and variance of the squared-errors computed across the stimulus set. Applied on a network with enough resources, the MV-BP learning algorithm yields learning curves similar to those observed with the standard backpropagation learning algorithm but with faster learning. When the new learning algorithm is used on a network with limited resources, learning is still faster, but performance asymptotes at a higher level of mean-square error. The proposed MV-BP learning algorithm might not find the best solution, but it is probably more adequate for modeling human cognitive learning since it allocates the resources in such a way that performance tends to be similar on all stimuli