Heavy traffic analysis of state-dependent parallel queues with triggers and an application to web search systems

In this paper, we consider the approximation of the number of customers of a Poisson-type parallel system of two queues, operating under heavy traffic, by a reflected stochastic differential equation. The type of queueing system here has the so-called fork-join structure. In addition, the model allows for state-dependent (service and arrival) rates. A novelty in our study is that a signal named “trigger” is considered. These signals are used to move a customer from one queue to the other. This, in turn, can be used to reduce the imbalance (and as a consequence, the system response time) which appears in certain parallel systems. The results here are applied to a web search system, which is a relevant example of a parallel system which operates under heavy traffic and suffers from imbalance among servers. A stochastic optimal control problem is formulated in order to find the best routing policy of signals which reduces the imbalance. The optimal control was tested in a simulation, indicating significant reduction in system response time when compared to the uncontrolled system.