RL-based optimal networked control considering network delay of discrete-time linear systems

Much attention has been paid to networked control systems where remote sensors, a controller, and remote actuators communicate via a network. During this communication, there is a network delay caused by data transmission. Unless this delay is considered, it may degrade the control performance and destabilize the plant in the worst case. In this paper, we consider a networked control system where the delays from distributed sensors to the controller differ from each other and fluctuate in a bounded range. We assume that all control inputs computed by the controller can be applied at the same time after they have been received by all actuators. Moreover, we consider a case where the parameters of the plant are unknown. We develop a reinforcement learning (RL)-based optimal controller that considers these delays and only needs the inputs and outputs. We propose a method to reconstruct the current state from the received output data from sensors and a method to learn the optimal gain using RL. The proposed controller can be applied to cases where the network delay has uncertainty. We perform a simulation to demonstrate the efficiency of the proposed controller.