A novel self-tuning CPS controller based on Q-learning method

This paper describes an application of Q-learning method based on-line self-tuning control methodology to solve the automatic generation control (AGC) under NERC´s new control performance standards (CPS). The AGC problem is a stochastic multistage decision problem, which can be modeled as a Markov decision process (MDP). This model-free Q-learning algorithm regards the CPS values as the rewards from the interconnected power systems. By regulating a closed-loop CPS control rule to maximize the total reward in the procedure of on-line learning, the optimal CPS control strategy can gradually obtained. The case study shows that after adding the Q-learning controller, the robustness and adaptability of AGC system is enhanced obviously and the CPS compliance is ensured.