Abstract :
In embedded control systems, the control input is computed based on sensing data of a plant in a processor and there is a delay, called the computation time delay, due to the computation and the data transmission. When we design an optimal controller, we need to take the delay into account to achieve its optimality. Moreover, in the case where it is difficult to identify a mathematical model of the plant, a model free approach is useful. Especially, the reinforcement learning-based approach has been much attention to in the design of an adaptive optimal controller. In this paper, we assume that the plant is a linear system but the parameters of the plant are unknown. Then, we apply the reinforcement learning to the design of an adaptive optimal digital controller with taking the computation time delay into consideration. First, we consider the case where all states of the plant are observed, and it takes L times to update the control input. An optimal feedback gain is learned from sequences of a pair of the state and the control input. Next, we consider the case where the control input is determined from outputs of the plant. We cannot use an observer to estimate the state of the plant since the parameters of the plant are unknown. So, we use a data-based control approach for the estimation. Finally, we apply the proposed adaptive optimal controller to attitude control of a quadrotor at the hovering state and show its efficiency by simulation.
Keywords :
adaptive control; control engineering computing; control system synthesis; data communication; delays; discrete time systems; embedded systems; feedback; learning (artificial intelligence); linear systems; optimal control; parameter estimation; state estimation; L-computation time delay; adaptive optimal digital controller; attitude control; data transmission; data-based control approach; embedded control systems; linear discrete-time systems; linear system; mathematical model; model free approach; optimal feedback gain; parameter estimation; reinforcement learning; Adaptation models; Delay effects; Optimal control; Output feedback; Propellers; State feedback;