Optimality results for a simple flow control problem

This paper presents a problem of optimal flow control for discrete-time M|M|1 queues, where the decision-maker seeks to maximize the throughput subject to a bound on the average queue size. The problem is cast as a constrained Markov decision process and solved via Lagrangian arguments. The optimal strategy is shown to be a threshold policy which saturates the constraint. The method of analysis proceeds through the discounted version of the Lagrangian problems whose value functions are shown to be integer-concave. Dynamic Programming and stochastic comparison ideas constitute the main ingredients of the solution.