Solving Multi-Agent Markov Decision Processes using learning automata

Multi-agent Markov decision processes (MMDPs) are widely used for modeling many types of multi-agent systems. In this paper, two new algorithms based on learning automata are proposed for solving MMDPs and finding optimal policies. In the proposed algorithms, Markov problem is described as a directed graph. The nodes of this graph are the states of the problem, and the directed edges represent the actions that result in transition from one state to another. Each node in the graph is equipped with a learning automaton and its actions are the outgoing edges of that node. Each agent moves from one node to another and tries to reach the goal state. In each node, the agent chooses its next transition with help of the learning automaton in that node. The actions taken by learning automata along the path traveled by the agent is then rewarded or penalized based on the cost of the traveled path according to a learning algorithm. This way the optimal policy for the agent will be gradually reached. The results of experiments have shown that our proposed algorithms perform better than the existing learning automata based algorithms in terms of cost and the speed of reaching the optimal policy.