Stochastic convergence analysis of the single-layer backpropagation algorithm for noisy input data

Author

Bershad, Neil J. ; Cubaud, Nicolas ; Shynk, John J.

Author_Institution

Dept. of Electr. & Comput. Eng., California Univ., Irvine, CA, USA

Volume

44

Issue

5

fYear

1996

fDate

5/1/1996 12:00:00 AM

Firstpage

1315

Lastpage

1319

Abstract

The statistical learning behavior of the single-layer backpropagation algorithm was analyzed for a system identification formulation for noise-free training data, transient and steady-state results were obtained for the mean weight behavior, mean-square error (MSE), and probability of correct classification. The article extends these results to the case of noisy training data, three new analytical results are obtained (1) the mean weights converge to finite values, (2) the MSE is bounded away from zero, and (3) the probability of correct classification does not converge to unity. However, over a wide range of signal-to-noise ratio (SNR), the noisy training data does not have a significant effect on the perceptron stationary points relative to the weight fluctuations. Hence, one concludes that noisy training data has a relatively small effect on the ability of the perceptron to learn the underlying weight vector F of the training signal model

Keywords

Gaussian noise; backpropagation; convergence of numerical methods; perceptrons; signal processing; statistical analysis; stochastic processes; MSE; SNR; correct classification probability; mean weight behavior; mean-square error; noise free training data; noisy input data; noisy training data; perceptron; signal-to-noise ratio; single layer backpropagation algorithm; statistical learning behavior; steady-state results; stochastic convergence analysis; system identification; training signal model; transient state results; weight fluctuations; Algorithm design and analysis; Backpropagation algorithms; Convergence; Signal to noise ratio; Statistical learning; Stochastic processes; Stochastic resonance; System identification; Training data; Transient analysis;

fLanguage

English

Journal_Title

Signal Processing, IEEE Transactions on

Publisher

ieee

ISSN

1053-587X

Type

jour

DOI

10.1109/78.502354

Filename

502354