Improving the training speed of three-layer feedforward neural nets by optimal estimation of the initial weights

The authors formulate the training problem for three-layer feedforward neural nets based on the well known linear algebra of D. Rumelhart et al. (1986). Then, they develop two estimation algorithms, called the forward estimation algorithm and the recurrent estimation algorithm, to estimate the initial weights. The basic idea is to set the initial weights space as close as possible to a global minimum before training, consequently reducing the training time. It is theoretically and empirically shown that a training procedure is unnecessary if the number of hidden units is equal to or greater than the number of training patterns minus one. Simulations were conducted for several problems. Results showed that the training speed is significantly improved by both initial weight estimation algorithms