Analysis on the Parameter of Back Propagation Algorithm with Three Weight Adjustment Structure for Hand Written Digit Recognition

Recently a commonly used method for Recognition of Handwritten Digit Application based on Back Propagation Neural Network (BPNN) has been widely applied. However, the original algorithm and its modifications contains a number of free parameters which affect particular networks differently and the slight error rate on the selection of these parameters can cause problems. Thus, this paper presents the effect of input parameters on BPNN with three different structures including Simple Back Propagation, Back Propagation with momentum terms and Back Propagation using conjugate gradient descent methods. To do so, this paper determined different parameters such as learning rate, momentum term or even the number of units in the hidden layer that exist in each structure. The data of UCI database is used for experiment in MATLAB program. The result showed that the Back Propagation with momentum term could perform very well leading to a recognition rate of 99%. The Simple algorithm obtained high recognition rate but it needed to increase learning rate, while Back Propagation using conjugate gradient descent could provide high result in case of improving hidden neural nodes. Thus, the result confirmed that adjustment of the relevant parameters are significant to obtain better recognition effect and higher accuracy.