Tradeoff between exploration and exploitation of OQ(λ) with non-Markovian update in dynamic environments

This paper presents some investigations on tradeoff between exploration and exploitation of opposition-based Q(lambda) with non-Markovian update (NOQ(lambda)) in a dynamic environment. In the previous work the authors applied NOQ(lambda) to the deterministic GridWorld problem. In this paper, we have implemented the NOQ(lambda) algorithm for a simple elevator control problem to test the behavior of the algorithm for non-deterministic and dynamic environment. We also extend the NOQ(lambda) algorithm by introducing the opposition weight to find a better tradeoff between exploration and exploitation for the NOQ(lambda) technique. The value of the opposition weight increases as the number of steps increases. Hence, it has more positive effects on the Q-value updates for opposite actions as the learning progresses. The performance of NOQ(lambda) method is compared with Q(lambda) technique. The experiments indicate that NOQ(lambda) performs better than Q(lambda).