Continuous-time restless bandit and dynamic scheduling for make-to-stock production

We study the "Whittle relaxation" version of the continuous time, discrete, and continuous state space Restless Bandit problem under the discounted cost criterion. Explicit expressions of Whittle\´s priority indexes, which generalize the Gittins indexes, are derived. This formalism is then used in the context of flexible make-to-stock production to construct dynamic scheduling rules. These analytical results are finally compared with the optimal numerically derived policy, obtained for a server delivering two product types. It is observed that the Whittle relaxation version of the Restless Bandit model nearly yields optimal dynamic scheduling rules.