An optimal scheduling policy for a two-flow priority queue with alternate paths

We consider a path-scheduling problem at a resource constrained node A that transmits two types of flows to a given destination through alternate paths. Type-1 flow is assumed to have a higher priority than type-2 flow thus, it is never rejected upon arrival. Type-2 flow, on the other hand, may be denied admission to the queue. Once accepted to the system, a packet joins queue 1 and is guaranteed service independent of its type. Instead of being rejected from service, packets have the option to be served at a slower server behind a second queue (queue 2) at node A. The slow server is intended mostly to serve low priority packets, therefore, type-1 packets are charged a switching cost in the event they are sent to queue 2. Transmitted packets receive a reward depending on which queue they were served at. The reward represents the resources saved for making that decision. A good path-scheduling policy at node A can reduce resource consumption at node A, extend the life of the efficient path, maximizes the service of both flows and guarantees the service of at least the high priority flow to the full extent. We propose and solve the path-scheduling problem for node A, which maximizes the average reward of successfully transmitting flows to a given sink, by dynamically assigning packets to one of the queues based on the packet type, the instantaneous queue lengths and the average reward for the associated path. We formulated the path-scheduling problem as a Markov decision process and show that the optimal policy is threshold-type.