Decomposition techniques for Markov zero-sum games with nested information

Markov zero-sum games arise in applications such as network interdiction, where an informed defender protects a network against attacks. This problem has received significant attention in recent years due to its relevance to military problems and network security. In this paper, we focus on finite games where the attacker knows imperfectly the network state, and formulate this as a Markov game with nested information. By exploiting the nested information structure, we decompose the multistage game into a sequence of one-stage subgames and develop an algorithm that computes the value of the game and the saddle point strategies for the game. This decomposition method computes the value of the game using backward induction as in stochastic dynamic programming, then identifies saddle-point strategies that achieve this value. Using the Markov structure of the game, we show that the value of the game can be computed efficiently in terms of a single value function of an information state at each stage. The resulting single stage optimization problems are much smaller than the original multistage game. We illustrate our results with an example of multistage network interdiction where the attacker may not be able to observe outcomes of the attacks.