Title :
A novel approach for training small-sized multilayer perceptrons
Author :
Chermakani, Deepak P.
Author_Institution :
Sch. of Electr. & Electron. Eng., Nanyang Technol. Univ., Singapore
Abstract :
The author present a novel approach tailored for training small-sized multilayer perceptrons. The approach presented updates a single randomly chosen weight with a stable second-order update rule, at each epoch. He discusses the analytical proof of stability and convergence at local minima. The paper identifies the situations when the approach is better than backpropagation for learning. Based on the experiments conducted, he shows four important results. First, the approach outperforms backpropagation when the network size is very small. Secondly, with slightly larger sized networks, though backpropagation tends to beat the approach when the perceptron activation function slope is low, the approach reaches better minima when the activation function slope is high. Thirdly, the approach normally reduces error when all the initial weights are equal, which is when back-propagation performs poorly. Finally, the approach tends to avoid premature saturation, during the early stages of learning, due to poorly initialized weights.
Keywords :
backpropagation; convergence; minimisation; multilayer perceptrons; activation function slope; backpropagation; convergence; error reduction; local minima; perceptron activation function; premature saturation avoidance; small sized multilayer perceptrons; stability; stable second order update rule; Biological system modeling; Biological systems; Error correction; Immune system; Iterative algorithms; Multilayer perceptrons; Neurons; Stability analysis;
Conference_Titel :
Neural Networks, 2003. Proceedings of the International Joint Conference on
Print_ISBN :
0-7803-7898-9
DOI :
10.1109/IJCNN.2003.1223714