Dynamic intrusion detection in resource-constrained cyber networks

We consider a large-scale cyber network with N components. Each component is either in a healthy state (0) or an abnormal state (1). Due to intrusions, the state of each component transits from 0 to 1 over time according to an arbitrary stochastic process. At each time, a subset of K (K <; N) components are probed and those observed in abnormal states are fixed. The objective is to design a dynamic probing strategy that minimizes the long-term network cost incurred at all abnormal components. We formulate the problem as a Restless Multi-Armed Bandit (RMAB) process. We show that this class of RMAB is indexable and Whittle index can be obtained in closed-form. For homogeneous networks, we show that Whittle index policy achieves the optimal performance with a simple structure that does not require any prior knowledge on the intrusion processes.