SPSA with state-dependent noise-a tool for direct adaptive control

The SPSA (simultaneous perturbation stochastic approximation) method for function minimization developed by Spall (1992) is analysed for state-dependent noise. We incorporate a restarting mechanism to ensure that the estimator sequence stays bounded. It is proved under certain conditions that the estimator sequence converges with rate O(n ^-β/2) for some β>0, where the rate is measured by the L_q-norm of the estimation error for any 1⩽q<∞. For the standard form of SPSA β can be 4/7-ε with any ε, as opposed to β=2/3 that has been obtained for state-dependent noise. Using higher order SPSA method the error exponent β can be made arbitrarily close to 1/2. The analysis of state-dependent noise is motivated by stochastic adaptive control, but two other examples will also be given