Optimal approximation schedules for iterative algorithms with application to dynamic programming

Many iterative algorithms rely on operators which may be difficult or impossible to evaluate exactly, but for which approximations are available. Furthermore, a graduated range of approximations may be available, inducing a functional relationship between computational complexity and approximation tolerance. In such a case, a reasonable strategy would be to vary tolerance over iterations, starting with a cruder approximation, then gradually decreasing tolerance as the solution is approached. In this article, it is shown that under general conditions, for linearly convergent algorithms the optimal choice of approximation tolerance convergence rate is the same linear convergence rate as the exact algorithm itself, regardless of the tolerance/complexity relationship. We illustrate this result by presenting a partial information value iteration (PIVI) algorithm for discrete time dynamic programming problems. The proposed algorithm makes use of increasingly accurate partial model information in order to decrease the computational burden of the standard value iteration algorithm. The algorithm is applied to a stochastic network example and compared to value iteration for the purpose of benchmarking.