Utilization of hierarchical structure stochastic automata for the back propagation method with momentum

Backpropagation (BP) is one of the most popular learning algorithms for multilayer networks, but it has limitations. Various modified BP methods have therefore been proposed. The BP method with momentum may be one of the most popular such modified algorithms. It has been reported that the BP method with momentum has been applied quite successfully to many practical problems. However, despite its effectiveness, this method involves the following serious problem: “Its learning performance depends heavily upon the selection of the value of momentum factor.” Unfortunately, it seems that there has not so far been proposed an intelligent algorithm for determining an appropriate value of the momentum factor. In this paper, we suggest that hierarchical structure stochastic automata are quite helpful for finding an appropriate value of the momentum factor of the BP method with momentum. Several computer simulation results confirm our idea