The negative effect on the control of inverted pendulum caused by the limit cycle in reinforcement learning

Control inverted pendulum is one of important applied regions of reinforcement learning. This paper analyzes negative effect on the control of inverted pendulum caused by the limit cycle. It points out the limit cycle will make Q-value converge to zero, and destroy the stabilization of the optimal control policy. Moreover higher degree of exploration can not overcome this problem, but rather intensify it. This paper discuss many solutions to this limit cycle, which succeed controlling inverted pendulum system and keep the stabilization of control policy