Optimal hop-by-hop flow control policies with multiple heterogeneous transmitters

The authors formulate the flow control between one receiving node and its adjacent transmitting nodes in a computer network as a Markovian decision process. The objective is to minimize the total number of messages waiting for transmission subject to the evolution of the state and to the constraints on the number of messages that can be received at the receiving node. The control action consists of an allocation of time slots among the transmitting nodes. The authors derive several structural properties which are useful in characterizing the optimal policy. Based on these properties, they show that the underlying M -dimensional optimality equation, where M stands for the number of transmitters, can be equivalently written as the one which involves a single state variable. The computational burden is thus greatly reduced