The measurement of strategy convergence for reinforcement learning in discrete state space

The concept of entropy is introduced into reinforcement learning. The definitions of the local strategy entropy and global strategy entropy are proposed. The global strategy entropy is proved to be the quantitative problem-independent measurement of the learning progress, i.e. the convergence degree of the strategy. To improve the learning performance, reinforcement learning with self-adaptive learning rate is proposed based on the strategy entropy. The experimental results show that learning based on the local strategy entropy has better learning performance than those with fixed learning rates.