Improving generalization performance by controlling α-information

The authors attempt to show that α-information, defined by an entropy of Renyi, is effective in improving the generalization performance. The α-information is represented in the difference between maximum Renyi entropy and an entropy, observed after finishing the learning. Thus, the information means the information content, stored in the network architecture. Improving the generalization performance corresponds to the adjustment of the information, stored in network architectures. For evaluating the performance of α-information, two problems of language acquisition: inference of regular verbs and irregular verb with grammatical determination were examined. In either case, the authors could clearly see that the generalization performance tended to be improved by changing α-information appropriately, especially as the network size was larger