Statistical analysis of the single-layer backpropagation algorithm for noisy training data

The statistical learning behavior of the single-layer backpropagation algorithm was analyzed using a system identification formulation for noise-free training data [Bershad et al. 1993]. Transient and steady-state results were obtained for the mean weight behavior, mean-square error (MSE), and probability of correct classification. The present paper extends these results to the case of noisy training data. Three new analytical results are obtained: 1) the mean weights converge to finite values even when the bias terms are zero, 2) the MSE is bounded away from zero, and 3) the probability of correct classification does not converge to unity. However, over a wide range of signal-to-noise ratios (SNRs), the noisy training data does not have a significant effect on the perceptron stationary points relative to the weight fluctuations. Hence, one concludes that noisy training data has a relatively small effect on the ability of the perceptron to learn the model weight vector F