Comparing gradient based learning methods for optimizing predictive neural networks

In this paper, we compare the performance of various gradient based techniques in optimizing the neural networks employed for prediction modeling. Training of neural network based predictive models is done using gradient based techniques, which involves searching for the point of minima on multidimensional energy function by providing step-wise corrective adjustment of weight vector present in hidden layers. Convergence of different gradient techniques is studied and compared by performing experiments in neural network toolbox package of MATLAB. Bulky data sets extracted from live data warehouse of life insurance sector are employed with gradient methods for developing the predictive models. Convergence behaviors of learning methods - gradient descent method, Levenberg Marquardt method, conjugate gradient method and scaled conjugate gradient method have been observed.