Title :
Projection-based methods for stepsize adaptation and their application to the training of feedforward artificial neural networks
Author :
Codrington, Craig W. ; Mohandes, Mohamed
Author_Institution :
Dept. of Electr. Eng., Purdue Univ., West Lafayette, IN, USA
fDate :
27 Jun-2 Jul 1994
Abstract :
Develops several adaptive step-size rules for gradient descent based on projecting weight and gradient vectors onto a set of unit vectors; each unit vector induces a one dimensional optimization problem which is solved by minimizing a fitted quadratic. A sum of squares criterion is then used to find the stepsize which which best fits the solution to each one dimensional optimization. The resulting stepsize rules are applied to train neural networks on parity problems of various sizes
Keywords :
feedforward neural nets; learning (artificial intelligence); optimisation; vectors; feedforward artificial neural networks; gradient descent; one dimensional optimization problem; parity problems; projection-based methods; stepsize adaptation; sum of squares criterion; training; Artificial neural networks; Backpropagation algorithms; Convergence; Error correction; Feedforward neural networks; Interpolation; Neural networks; Size measurement; Stability; Time measurement;
Conference_Titel :
Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on
Conference_Location :
Orlando, FL
Print_ISBN :
0-7803-1901-X
DOI :
10.1109/ICNN.1994.374141
