Deadline-constrained maximum reliability packet forwarding with limited channel state information

This paper considers real-time packet forwarding over wireless multi-hop networks with lossy and bursty links. Our objective is to maximize the probability that individual packets reach their destination before a hard deadline. The loss processes on links are modeled by finite-state Markov chains. While the parameters of the Markov chains are assumed to be known, the instantaneous channel states are not accessible but have to be estimated from observations of successes and failures of actual packet transmissions. We formulate the forwarding problem as a partially observable Markov decision process and derive the optimal forwarding policy. A novel technique, based on maximum-volume inscribed ellipsoids, for computing approximate solutions with reduced implementation complexity is proposed. We further discuss structural properties of the value function and the optimal actions. Finally, numerical examples illustrate the power of the developed techniques.