Optimal Defense and Control for Cyber-Physical Systems

In this paper, we present a novel representation for cyber-physical systems wherein the states of the physical system are incorporated into the cyber system and vice versa. Next, by using this representation, optimal strategies are derived for the defender and the attacker by using zero-sum game formulation and iterative Q-learning is utilized to obtain the Nash equilibrium. In addition, a Q-learning-based optimal controller is revisited for the physical system in the presence of uncertain dynamics resulting from the cyber system under attacks. The benefit of the learning strategy is that the approach can handle a variety of attacks provided they affect packet losses and delays. Simulation results, on the yaw-channel control of the unmanned aerial vehicle (UAV), show that on the cyber side, both the defender and the attacker gain their largest payoff and on the physical system side, the optimal controller maintains the system stable.