Training MLPs layer-by-layer with the information potential

In the area of information processing one fundamental issue is how to measure the statistical relationship between two variables based only on their samples. The authors previously (1998) presented the idea of information potential which was formulated from the quadratic mutual information, and successfully applied it to problems such as blind source separation and pose estimation of SAR images. This paper shows how information potential can be used to train a MLP (multilayer perceptron) layer-by-layer, which provides evidence that the hidden layer of a MLP serves as an “information filter” which tries to best represent the desired output in that layer in the statistical sense of mutual information