Discounted Optimality for Continuous-Time Markov Decision Processes in Polish Spaces

This paper deals with continuous-time Markov decision processes in Polish spaces, under a discounted expected reward criterion. We first give conditions under which we ensure the existence of optimal stationary policies by using the technique of extended infinitesimal operators, and also provide a recursive way to compute (or at least to approximate) the optimal reward values. Finally, we use controlled generalized Potlach processes to show that all conditions in this paper are satisfied, whereas the earlier conditions fail to hold, and then we further apply our results to the discounted optimal control problems of several population processes.