Intermittently Proving Dynamic Programming to Solve Infinite MDPs on GPUs

In this paper, we propose a variant of the dynamic programming which is suitable for solving infinite Markov decision processes on GPUs. The primary feature of the proposed method is to not always but intermittently transfer and check values for proving the convergence of the procedure. It is expected for the proposed method to decrease computational times by suppressing surplus transfers and checks of values. This expectation is verified through applications of some dynamic programming programs to a simple animat problem and the mountain-car problem.