Learning with Eligibility Traces in Adaptive Critic Designs

In this paper, we study the training strategies of the critic network in adaptive critic designs. The conventional strategy is always to conduct an internal training cycle for the specified object at each time step based on relative information in two consecutive moments. Whereas, in our work, the mechanism of eligibility traces is adopted for the learning of the cost function or its derivatives. The new learning depends on the current error combined with traces of past events. For the classical single cart-pole balancing problem, we implement our idea with one typical adaptive critic design, i.e. action-dependent heuristic dynamic programming. And comparing results demonstrate our approach with more efficiency in the performances such as learning speed and success rate of learning.