Error back propagation with minimum-entropy weights: a technique for better generalization of 2-D shift-invariant NNs

For better generalization of shift-invariant neural networks, the authors propose a modified backpropagation learning rule, which reduces the complexity of the neural network as a whole, instead of removing the particular hidden units. The measure of the complexity of the neural network is defined as the entropy of the connectivity pattern. Learning in the neural network is carried out to minimize this measure as well as the output error. An example of line-feature detection using a shift-invariant neural network is presented. Simulation results show that the neural network response function is generalized by the modified learning rule to be independent of the angle of lines, after being trained at some discrete angles