Advantage updating on a mobile robot

The paper describes the first experiments with advantage updating on a mobile robot. Advantage updating is an algorithm for reinforcement learning that has been recently developed and only simulation studies have been done previously. The task for the mobile robot was to move forward for a period of a few seconds if possible and to avoid obstacles otherwise using sonar, tactile, and motor position sensors. The performance of advantage updating was compared to the related Q-learning algorithm. While both algorithms were able to solve a one-dimensional form of the problem with only 2 states and 2 actions, advantage updating outperformed Q-learning for the two-dimensional problem with 4 actions. This was due to advantage updating´s explicit account of time and the use of a variable time step for updating.