Bayesian congestion control over a Markovian network bandwidth process

We formulate a Bayesian congestion control problem in which a source must select the transmission rate over a network whose available bandwidth is modeled as a time-homogeneous finite-state Markov Chain. The decision to transmit at a rate below the instantaneous available bandwidth results in an under-utilization of the resource while transmission at rates higher than the available bandwidth results in a linear penalty. The trade-off is further complicated by the asymmetry in the information acquisition process: transmission rates that happen to be larger than the instantaneous available bandwidth result in perfect observation of the state of the bandwidth process. In contrast, when transmission rate is below the instantaneous available bandwidth, only a (potentially rather loose) lower bound on the available bandwidth is revealed. We show that the problem of maximizing the throughput of the source while avoiding congestion loss can be expressed as a Partially Observable Markov Decision Process (POMDP). We prove structural results providing bounds on the optimal actions. The obtained bounds yield tractable sub-optimal solutions that are shown via simulations to perform well.