Combining reinforcement learning with GA to find co-ordinated control rules for multi-agent system

In a multi-agent application, it is necessary to find co-ordinated control rules that maximise a global objective function. To establish coordination, a real-time synchronous communication is normally assumed. However, communication is often limited to asynchronous and very time delayed methods in many practical applications. The paper intends to propose a method to search for co-ordinated plans under limited information exchange. Our approach is to combine on-line local optimisation by reinforcement learning (RL) and asynchronous global combinatorial optimisation by genetic algorithms. The GA search modifies RL´s search direction to find a co-ordinated plan, whereas the RL tries to obtain that plan in real-time. Information on which direction is better to find by RL is given through long-term (not instant) communication. The direction is given by a state compression mapping. This is therefore a Lamarckian type GA that inherits acquired knowledge from RL. By using a seesaw balancing problem as a test bed, the performance of the proposed method is shown.