Optimal control of observable continuous time Markov chains

This paper considers the optimal control of time varying, finite horizon, continuous time Markov chains under the assumption that their behavior can be influenced by the adjustment of selected transition rates. We assume a quadratic penalty on the amount of the rate adjustment and that the system is completely observable. We derive an ordinary differential equation whose solution gives the minimum return function and describe how the optimal feedback control law is obtained from it. The results bear some resemblance to the solution of the quadratic regulator problem for linear systems, but because of the bilinear structure of these problems, the details are significantly different.