DocumentCode :
1922889
Title :
A novel approach for training small-sized multilayer perceptrons
Author :
Chermakani, Deepak P.
Author_Institution :
Sch. of Electr. & Electron. Eng., Nanyang Technol. Univ., Singapore
Volume :
3
fYear :
2003
fDate :
20-24 July 2003
Firstpage :
1999
Abstract :
The author present a novel approach tailored for training small-sized multilayer perceptrons. The approach presented updates a single randomly chosen weight with a stable second-order update rule, at each epoch. He discusses the analytical proof of stability and convergence at local minima. The paper identifies the situations when the approach is better than backpropagation for learning. Based on the experiments conducted, he shows four important results. First, the approach outperforms backpropagation when the network size is very small. Secondly, with slightly larger sized networks, though backpropagation tends to beat the approach when the perceptron activation function slope is low, the approach reaches better minima when the activation function slope is high. Thirdly, the approach normally reduces error when all the initial weights are equal, which is when back-propagation performs poorly. Finally, the approach tends to avoid premature saturation, during the early stages of learning, due to poorly initialized weights.
Keywords :
backpropagation; convergence; minimisation; multilayer perceptrons; activation function slope; backpropagation; convergence; error reduction; local minima; perceptron activation function; premature saturation avoidance; small sized multilayer perceptrons; stability; stable second order update rule; Biological system modeling; Biological systems; Error correction; Immune system; Iterative algorithms; Multilayer perceptrons; Neurons; Stability analysis;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Neural Networks, 2003. Proceedings of the International Joint Conference on
ISSN :
1098-7576
Print_ISBN :
0-7803-7898-9
Type :
conf
DOI :
10.1109/IJCNN.2003.1223714
Filename :
1223714
Link To Document :
بازگشت