Continuous-time Markov decision processes with th-bias optimality criteria

In this paper, we study the n th-bias optimality problem for finite continuous-time Markov decision processes (MDPs) with a multichain structure. We first provide n th-bias difference formulas for two policies and present some interesting characterizations of an n th-bias optimal policy by using these difference formulas. Then, we prove the existence of an n th-bias optimal policy by using n th-bias optimal policy iteration algorithms, and show that such an n th-bias optimal policy can be obtained in a finite number of policy iterations.