Attention allocation for decision making queues

We consider the optimal servicing of a queue with sigmoid server performance. There are various systems with sigmoid server performance, including systems involving human decision making, visual perception, human–machine communication and advertising response. Tasks arrive at the server according to a Poisson process. Each task has a deadline that is incorporated as a latency penalty. We investigate the trade-off between the reward obtained by processing the current task and the penalty incurred due to the tasks waiting in the queue. We study this optimization problem in a Markov decision process (MDP) framework. We characterize the properties of the optimal policy for the MDP and show that the optimal policy may drop some tasks; that is, may not process a task at all. We determine an approximate solution to the MDP using the certainty-equivalent receding horizon optimization framework and derive performance bounds on the proposed receding horizon policy. We also suggest guidelines for the design of such queues.