Fault tolerant learning using Kullback-Leibler divergence

In this paper, an objective function for training a fault tolerant neural network is derived based on the idea of Kullback-Leibler (KL) divergence. The new objective function is then applied to a radial basis function (RBF) network that is with multiplicative weight noise. Simulation results have demonstrated that the RBF network trained in accordance with the new objective function is of better fault tolerance ability, in compared with the one trained by explicit regularization. As KL divergence has relation to Bayesian learning, a discussion on the proposed objective function and the other Bayesian type objective functions is discussed.