Incremental least squares policy iteration in reinforcement learning for control

Author

Li, Chun-Gui ; Wang, Meng ; Yang, Shu-Hong

Author_Institution

Dept. of Comput. Eng., Guangxi Univ. of Technol., Liuzhou

Volume

4

fYear

2008

fDate

12-15 July 2008

Firstpage

2010

Lastpage

2014

Abstract

We propose a novel algorithm of reinforcement learning for control problems which combines value-function approximation with linear architectures and approximate policy iteration. This algorithm improves least-squares policy iteration (LSPI) methods by using incremental least-squares temporal-difference learning algorithm (iLSTD) for prediction problems. We show that the novel algorithm has less computing complexities than LSPI, and has the same performance as LSPI in learning optimal policies.

Keywords

adaptive control; iterative methods; learning (artificial intelligence); learning systems; least squares approximations; approximate policy iteration; incremental least squares policy iteration; incremental least-squares temporal-difference learning algorithm; linear architectures; prediction problems; reinforcement learning; value-function approximation; Computer architecture; Convergence; Cybernetics; Electronic mail; Function approximation; Least squares approximation; Least squares methods; Linear approximation; Machine learning; Scheduling algorithm; Linear function approximation; incremental updating; least-squares methods; policy evaluation; policy iteration;

fLanguage

English

Publisher

ieee

Conference_Titel

Machine Learning and Cybernetics, 2008 International Conference on

Conference_Location

Kunming

Print_ISBN

978-1-4244-2095-7

Electronic_ISBN

978-1-4244-2096-4

Type

conf

DOI

10.1109/ICMLC.2008.4620736

Filename

4620736