Fault tolerance in artificial neural networks

Different strategies for overcoming hardware failures in artificial neural networks are presented. The failure of one or more units in the hidden layer of layered feedforward networks is especially addressed. Different types of retraining techniques are investigated, and required retraining efforts are correlated with the internal representations for specific classification tasks. Subsequently, a practical technique is introduced to achieve true fault tolerance, i.e., to have the network continue to function correctly after failure of one or more hidden units. To achieve this fault-tolerant behavior, hidden units are randomly disabled for some pattern presentations during a standard backpropagation training phase. Prolonged training in this mode can achieve fault tolerance even with respect to fault patterns for which the network is not specifically trained