Mutual information neural networks: a new connectionist approach for dynamic speech recognition tasks

A new probabilistic neural network paradigm for dynamic pattern recognition problems is presented. The approach includes the following innovations: 1) it is based on a self-organizing learning approach using information theory principles. 2) The neuron activations are interpreted as probabilities and represent probabilistic decision boundaries in the feature space. 3) A combination of unsupervised and supervised learning algorithms is used to train the network weights. 4) The neuron probabilities can be further refined by corrective training methods. 5) The neural network can process dynamic patterns of arbitrary length, and can be even used for continuous speech recognition, although it is not a recurrent network. 6) The output activations of the neural network can be evaluated directly or optionally treated as input to hidden Markov models in order to construct a hybrid recognition system. The network has been tested for the recognition of dynamic speech patterns and performs better than a discrete HMM system with a codebook size equal to the number of output neurons in the neural net