Equivalence between Weight Decay Learning and Explicit Regularization to Improve Fault Tolerance of RBF

Although weight decay learning has been proposed to improve generalization ability of a neural network, many simulated studies have demonstrated that it is able to improve fault tolerance. To explain the underlying reason, this paper presents an analytical result showing the equivalence between adding weight decay and adding explicit regularization on training a RBF to tolerate multiplicative weight noise. Under a mild condition, it is proved that explicit regularization will be reduced to weight decay.