Event-triggered optimal regulation of uncertain linear discrete-time systems by using Q-learning scheme

In this paper, an event-triggered optimal adaptive regulation of an uncertain linear discrete time system is proposed. This scheme solves the optimal control in a forwardin- time and online manner by using both dynamic programming and Q learning. First, the time varying action dependent value or the Q-function is estimated online by an adaptive value function estimator (VFE) with event-based state vector and a time dependent basis function. The estimated value function parameters are subsequently used to generate the optimal control gain matrix. Further, aperiodic tuning law for the VFE parameters is proposed not only to estimate the parameters but also handle the terminal constraint. The parameters are tuned only at the event-trigger instants thus reducing computation when compared to the traditional optimal adaptive control. Above all, an adaptive event-trigger condition to decide the event-trigger instants and guarantee stability of the closed-loop system is analytically derived based on the optimal performance criterion via Lyapunov direct method. Nonetheless, the existence of a non-trivial minimum inter-event time is analyzed. Further, it is shown that the parameters converge asymptotically provided the persistency of excitation condition on the regression vector is ensured. Finally, the analytical design is validated with the simulation results.