Controlling the input/output function of the neural network by variable offset rule

A sigmoid function has been utilized for the input/output functions of the backpropagation type neural networks. It, however, has a local minimum problem; if the output of the sigmoid function becomes 0 or 1, no further learning occurs even if there are errors between teaching inputs and outputs of the output unit. The offset method of applying some offset values to the intermediate layer cells is thought to be effective in solving the local minimum problem. In this paper, we propose two formulations of offset function: the linearly decremental offset function that decrements offset value as iteration of the learning process increases, and the logarithmic error offset function that varies offset values according as logarithm of output errors. The performance of these methods are evaluated by recognition test of handwriting.