Comparisons of continuous-time and discrete-time Q-learning schemes for adaptive linear quadratic control

Author

Chun, Tae Yoon ; Lee, Jae Young ; Park, Jin Bae ; Choi, Yoon Ho

Author_Institution

Dept. of Electr. & Electron. Eng., Yonsei Univ., Seoul, South Korea

fYear

2012

fDate

20-23 Aug. 2012

Firstpage

1228

Lastpage

1233

Abstract

In this paper, we compare two online model-free Q-learning schemes for adaptive linear quadratic (LQ) control of discrete-time (DT) and continuous-time (CT) dynamical systems. Both Q-learning schemes come from the optimality principles, but the DT and CT Q-learning is designed with different Q-functions. This difference may results in the different exploration properties and convergence speeds. Numerical simulations with an ideal DC motor are carried out to further investigate and compare the Q-learning methods.

Keywords

DC motors; adaptive control; continuous time systems; convergence; discrete time systems; learning systems; linear quadratic control; numerical analysis; CT Q-learning; CT dynamical system; DC motor; DT Q-learning; DT dynamical system; LQ control; Q-function; adaptive linear quadratic control; continuous-time Q-learning scheme; continuous-time dynamical system; convergence speed; discrete-time Q-learning scheme; discrete-time dynamical system; numerical simulation; online model-free Q-learning scheme; optimality principle; Convergence; DC motors; Educational institutions; Equations; Heuristic algorithms; Numerical simulation; Vectors; LQR; g-learning; policy iteration; reinforcement learning;

fLanguage

English

Publisher

ieee

Conference_Titel

SICE Annual Conference (SICE), 2012 Proceedings of

Conference_Location

Akita

ISSN

pending

Print_ISBN

978-1-4673-2259-1

Type

conf

Filename

6318633