On the influence of the number of layers on the performance and convergence behavior of the back propagation algorithm

In many neural network applications to signal processing, the back propagation (BP) algorithm is used for the training process. Recently, several authors have analyzed the behavior of the BP algorithm and studied its properties. The influence of the number of layers on the performance and convergence behavior of the BP algorithm remains, however, not well known. The paper tries to investigate this problem by studying a simplified multilayer neural network used for adaptive filtering. The analysis is based upon the derivation of recursions for the mean weight update which can be used to predict the weights and mean squared error over time. The paper shows also the effects of the algorithm step size and the initial weight values upon the algorithm behavior. Computer simulations display good agreement between the actual behavior and the predictions of the theoretical model. The properties of the BP algorithm are illustrated through several simulation examples and compared to the classical LMS algorithm