Optimal hop-by-hop flow control policies in computer networks with multiple transmitters: convexity and monotonicity properties. I. linear and equal holding costs

The problem of flow control between one receiving node and its adjacent transmitting nodes in a computer network is modeled as a Markov decision problem. Given that the control action is to allocate a fixed number of time slots among M⩾2 transmitters, the objective is to characterize policies that minimize the total number of messages awaiting service at the transmitting nodes subject to the evolution of the state. The authors partially characterize a set of optimal policies and show that this characterization reduces the problem of finding all the state-dependent optimal policies to one of finding a finite number. Moreover, the number of policy computations required to find the optimal ones is reduced significantly. For M=2 the optimal policy is a monotone function of the state and the total cost is convex. This convexity property of the total cost makes it possible to characterize the optimal control policy further when the process of message generation at one transmitter is stochastically larger than at the other. When the processes of message generation at the M transmitters are independent and identically distributed, the explicit form of the optimal control is found