Optimal channel probing in communication systems: The two-channel case

We consider a multi-channel communication system in which a transmitter has access to two channels, but does not know the state of either channel. We model the channel state using an ON/OFF Markovian model, and allow the transmitter to probe one of the channels at predetermined probing intervals to decide over which channel to transmit. For models in which the transmitter must transmit over the probed channel, it has been shown that a myopic policy that probes the channel most likely to be ON is optimal. In this work, we allow the transmitter to select a channel over which to transmit that is not necessarily the one it probed. We show that in the case where the two channels are i.i.d, all probing policies yield equal reward. We extend this problem to dynamically choose when to probe based on the results of previous probes, and characterize the optimal policy, as well as provide a LP in terms of state action frequencies to find the optimal policy.