An Improved Algorithm for Eleman Neural Network to Avoid the Local Minima Problem

Eleman Neural Network has been widely used in various fields, from classification to the prediction categories in natural language data. However, the local minima problem usually occurs in the process of the learning. To solve this problem and to speed up the process of the convergence, we propose an improved learning method by adding a term in error function which relates to the neuron saturation of the hidden layer for the Eleman Neural Network. The activation functions are adapted to prevent neurons in the hidden layer from getting into deep saturation area. We apply this method to the Boolean Series Prediction Questions to demonstrate its validity. The simulation result shows that the proposed algorithm can avoid the local minima problem, largely accelerate the speed of the convergence and get good results for the simulation tasks.