A study on the simple penalty term to the error function from the viewpoint of fault tolerant training

We discussed training algorithm for multi-layer neural networks to enhance fault tolerance of the trained networks. In our previous paper, we proposed adding a simple penalty term to the error function for BP algorithm. The penalty term is a simple polynomial (sum of n-th power of weights). It is also introduced for another purpose (structural training). In this paper, we discuss about the effect of the term, especially the effect of its exponent. Through some experiments and discussions, we conclude that the change of the parameter n brings drastic change of its effect. For small n, the training works as the structural training. For large n, the training enhances the fault tolerance of trained networks.