Task decomposition and dynamic policy merging in the distributed Q-learning classifier system

A distributed reinforcement learning system is designed and implemented on a mobile robot for the study of complex task decomposition and dynamic policy merging in real robot learning environments. The distributed Q-learning classifier system (DBLCS) is evolved from the standard LCS proposed by Holland (1996). We address two of the limitations of the LCS through the use of Q-learning as the apportionment of credit component and a distributed learning architecture to facilitate complex task decomposition. The Q-learning update equation is derived and its advantages over the complex bucket brigade algorithm (BBA) are discussed. Holistic and monolithic shaping approaches are used to distribute reward among the learning modules of the DBLCS and allow dynamic policy merging in a variety of real robot learning experiments