Gradient methods in unsupervised neural networks

The authors discuss the application of gradient methods (steepest descent and conjugate gradients) to an unsupervised neural network that uses potential functions in the output layer as previously reported by the authors (1990). The nature of most unsupervised neural networks makes the gradient step algorithm the only possible way to optimize the network´s function. The network reported by the authors in 1990 can make use of other gradient methods. Experiments with the steepest descent/ascent and conjugate gradients algorithms show a potential decrease in the number of iterations. The contours of the network´s function showed considerable variation, especially close to the local optima. This nature restricted the line search algorithms that could be used. Simulations confirm this fact by showing how a crude line search algorithm performs better than a semiexact line search algorithm