Online optimal data transmission strategy under Markov channel

The challenges of providing data services over wireless networks come from, their high reliability requirement under noisy wireless channels. In this paper, we model the noisy channel by a continuous-time Markov process and propose a dynamical packet transmission strategy for data service. The transmission strategy optimization is formulated as a Markov decision process. We establish that the optimal strategy has a threshold structure and develop a single sample path-based optimization algorithm to tune the thresholds. Since only a single sample path is involved, the proposed algorithm can be implemented online.