Effective perturbation distributions for small samples in simultaneous perturbation stochastic approximation

Simultaneous perturbation stochastic approximation (SPSA) has proven to be an efficient algorithm for recursive optimization. SPSA uses a centered difference approximation to the gradient based on only two function evaluations regardless of the dimension of the problem. Typically, the Bernoulli ±1 distribution is used for the perturbation vector and theory has been established to prove the asymptotic optimality of this distribution. However, efficiency of the Bernoulli distribution may not be guaranteed for small-sample approximations. In this paper, we investigate the performance of segmented uniform distribution for the perturbation vector. For small-sample approximations, we show that the Bernoulli distribution may not be the best for a certain choice of parameters.