Exponential moving average Q-learning algorithm

Author

Awheda, Mostafa D. ; Schwartz, Howard M.

Author_Institution

Dept. of Syst. & Comput. Eng., Carleton Univ., Ottawa, ON, Canada

fYear

2013

fDate

16-19 April 2013

Firstpage

31

Lastpage

38

Abstract

A multi-agent policy iteration learning algorithm is proposed in this work. The Exponential Moving Average (EMA) mechanism is used to update the policy for a Q-learning agent so that it converges to an optimal policy against the policies of the other agents. The proposed EMA Q-learning algorithm is examined on a variety of matrix and stochastic games. Simulation results show that the proposed algorithm converges in a wider variety of situations than state-of-the-art multi-agent reinforcement learning (MARL) algorithms.

Keywords

iterative methods; learning (artificial intelligence); matrix algebra; moving average processes; multi-agent systems; stochastic games; EMA Q-learning algorithm; EMA mechanism; MARL algorithms; Q-learning agent; exponential moving average Q-learning algorithm; multiagent policy iteration learning algorithm; multiagent reinforcement learning algorithms; optimal policy; stochastic games; Games; Heuristic algorithms; Learning (artificial intelligence); Markov processes; Nash equilibrium; Probability distribution; Vectors;

fLanguage

English

Publisher

ieee

Conference_Titel

Adaptive Dynamic Programming And Reinforcement Learning (ADPRL), 2013 IEEE Symposium on

Conference_Location

Singapore

ISSN

2325-1824

Type

conf

DOI

10.1109/ADPRL.2013.6614986

Filename

6614986