Information maximization for feature detection and pattern classification by autoencoders

Proposes an information maximization method, which can be used for feature detection and pattern classification. The information is defined by the decrease of the uncertainty, measured by the entropy of hidden units. By maximizing the information, hidden units tend to respond specifically to input patterns. Features, common to all the input patterns and specific to input patterns, are gradually separated. Some important or kernel components in network architectures are generated, which enable one to interpret easily the basic mechanism of the network behaviors. The authors applied the information maximization method to the feature detection and the classification of phonemes by using autoencoders which reproduce exactly input patterns at output units. The authors could observe that networks could detect features of phonemes explicitly and classify appropriately input patterns by maximizing the information. The main principle of the feature detection by the information maximization consists in the detection of features, common to many phonemes. Based upon detected common features, appropriate distinctive features can be selected