On conditions for convergence rates of stochastic approximation algorithms

We develop deterministic necessary and sufficient conditions on individual noise sequences of a stochastic approximation algorithm for the error of the iterates to converge at a given rate. Specifically, suppose {p_n} is a given positive sequence converging monotonically to 0. Consider a stochastic approximation algorithm x_n+1=x_n-a_n(A_nx_n-b _n)+a_ne_n, where {x_n} is the iterate sequence, {a_n} is the step size sequence, {e_n } is the noise sequence, and x* is the desired zero of the function f(x)=Ax-b. We show that x_n-x*=o(ρ_n) if and only if the sequence {e_n} satisfies one of five equivalent conditions. These conditions are based on well known formulas for noise sequences found in the literature