Channel probing for opportunistic access with multi-channel sensing

We consider an opportunistic communication system consisting of multiple independent channels with time-varying states. We formulate the problem of optimal sequential channel selection as a restless multi-armed bandit process, for which a powerful policy-Whittle´s index policy-can be implemented based on the indexability of the system. We obtain Whittle´s index in closed-form under the average reward criterion, which leads to the direct implementation of Whittle´s index policy. To evaluate the performance of Whittle´s index policy, we provide simple algorithms to calculate an upper bound of the optimal performance. The tightness of the upper bound and the near-optimal performance of Whittle´s index policy are illustrated with simulation examples. When channels are stochastically identical, we show that Whittle´s index policy is equivalent to the myopic policy, which has a simple and robust structure. Based on this structure, we establish the approximation factors of the performance of Whittle´s index policy. Furthermore, we show that Whittle´s index policy is optimal under certain conditions.