Training techniques to obtain fault-tolerant neural networks

This paper addresses methods of improving the fault tolerance of feedforward neural nets. The first method is to coerce weights to have low magnitudes during the backpropagation training process, since fault tolerance is degraded by the use of high magnitude weights; at the same time, additional hidden nodes are added dynamically to the network to ensure that desired performance can be obtained. The second method is to add artificial faults to various components (nodes and links) of a network: during training. The third method is to repeatedly remove nodes that do not significantly affect the network: output, and then add new nodes that share the load of the more critical nodes in the network. Experimental results have shown that these methods can obtain better robustness than backpropagation training, and compare favorably with other approaches.<>