Stochastic approximation with averaging and feedback: rapidly convergent “on-line” algorithms

Consider the stochastic approximation X_n+1=X_n+a_ng(X_n, ξ_n ), where 0<a_n→0, Σ_n a_n =∞ and {ξ_n} is the “noise” sequence. Suppose that a_n→0 slowly enough such that a _n/a_n+1=1+o(a_n). Then various authors have shown that the rate of convergence of the average X¯_n =1/n Σ_i=1ⁿ X_i is optimal in the sense that √n(X¯_n-θ) converged in distribution to a normal random variable with mean zero and covariance V, where V was the smallest possible in an appropriate sense. V did not depend on {a_n}. The analogs of the advantages of averaging extend to the constant parameter systems X_n+1=X_n+εg(X_n, ξ_n) for small ε>0. The averaging method is essentially “off line” in the sense that the actual SA iterate X_n is not influenced by the averaging. In many applications, X_n itself is of greatest interest, since that is the “operating parameter”. This paper deals with the problem of stochastic approximation with averaging and with appropriate feedback of the averages into the original algorithm. It is shown both mathematically and via simulation that it works very well and has numerous advantages. It is a clear improvement over the system X_n by itself. It is fairly robust, and quite often it is much preferable to the use of the above averages without feedback. The authors deal, in particular, with “linear” algorithms of the type appearing in parameter estimators, adaptive noise cancellers, channel equalizers, adaptive control, and similar applications. The main development is for the constant parameter case because of its importance in applications. But analogous results hold for the case where a_n→0