A stochastic approximation method for noise-free optimization

Author

Gerencser, Laszlo ; Vago, Zsuzsanna

Author_Institution

Comput. & Autom. Inst., Budapest, Hungary

fYear

2001

fDate

4-7 Sept. 2001

Firstpage

1496

Lastpage

1500

Abstract

The simultaneous perturbation stochastic approximation or SPSA method for function minimization developed in (Spall, 1999) is analyzed for optimization problems without measurement noise. We prove that, under appropriate technical conditions, the estimator sequence converges to the optimum with geometric rate with probability 1. Numerical experiments are carried out to determine the top Lyapunov-exponent. We conclude that randomization improves convergence rate while significantly reducing the number of function evaluations.

Keywords

approximation theory; minimisation; perturbation techniques; probability; random processes; recursive estimation; stochastic processes; Lyapunov-exponent; SPSA method; estimator sequence; function evaluations; function minimization; geometric rate; noise-free optimization; probability; randomization; simultaneous perturbation stochastic approximation; Approximation methods; Convergence; Europe; Minimization; Optimization; Recursive estimation; Stability analysis; Kiefer-Wolfowitz-methods; Lyapunov-exponents; Optimization; recursive estimation; stochastic approximation;

fLanguage

English

Publisher

ieee

Conference_Titel

Control Conference (ECC), 2001 European

Conference_Location

Porto

Print_ISBN

978-3-9524173-6-2

Type

conf

Filename

7076130