Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm

It is known that a stochastic approximation (SA) analogue of the deterministic Newton-Raphson algorithm provides an asymptotically optimal or near-optimal form of stochastic search. In a recent paper, Spall (2006) introduces two enhancements that generally improve the quality of the estimates for underlying Jacobian (Hessian) matrices, thereby improving the quality of the estimates for the primary parameters of interest. The first enhancement rests on a feedback process that uses previous Jacobian estimates to reduce the error in the current estimate. The second enhancement is based on the formation of an optimal weighting of "per-iteration" Jacobian estimates. This paper provides a formal convergence analysis for the algorithm introduced in Spall (2006). In particular, we present conditions for the almost sure convergence of the Jacobian estimates with the feedback and weighting. We also develop results for the rate of convergence in both the noisy and noise-free settings.