Mutual information training and size minimization of adaptive probabilistic neural networks

An upper bound of the Bayes error probability is used as a generalized performance criterion for supervised neural network classifiers. It is shown that the maximization of the mutual information is equivalent to the minimization of this bound, which leads to a direct implementation of the Bayes framework for classification. The criterion is used both in training neural networks and in minimizing their size by adaptive pruning. A top-down heuristic for adaptively pruning oversized networks is proposed and applied. The approach is applied to adaptive probabilistic neural networks. Two benchmark problem results are given, verifying its validity