Quasi stochastic approximation

In recent work it was shown that a deterministic analog of stochastic approximation can be formulated to obtain a Q-learning algorithm for approximate optimal control of deterministic and stochastic systems. This paper provides a general foundation for "quasi-stochastic approximation" in which all of the processes under consideration are deterministic, much like quasi-Monte-Carlo for variance reduction in simulation. Applications to root finding and to TD-learning are described, and numerical results are presented.