Convergence analysis of recurrent neural network with self-loops based on eigenvalues of a connection matrix

Recurrent neural networks (RNNs) can be applied to solve a combinatorial optimization problem. However, the existence of the local minima in this model prevents its application to real world. In this paper, an analysis method for network dynamics based on eigenvalues and eigenvectors of a connection weight matrix is proposed. In this analysis, the transition of the number of negative eigenvalues and the movement of eigenspaces by increasing diagonal elements, which correspond to feedback loops of RNNs, are discussed. From this analysis, it is confirmed that the number of negative eigenvalues decreases and the eigenspaces move toward the ascent side of the energy slope at the center point of the state space of RNNs, as increasing diagonal elements. These behaviors of RNNs contribute to the improvement for searching a solution of a combinatorial optimization problem. This analysis method is applied to a 2-dimensional example and five cities TS problems. From this analysis, it is theoretically found that the network performance of detecting the optimal solution can be improved by increasing the values of diagonal elements of connection matrix