Stochastic approximation algorithms for the local optimization of functions with nonunique stationary points

The aim of this paper is the provision of a framework for a practical stochastic unconstrained optimization theory. The results are based on certain concepts of stochastic approximation, although not restricted to those procedures, and aim at incorporating the great flexibility of currently available deterministic optimization ideas into the stochastic problem, whenever optimization must be done by Monte Carlo or sampling methods. Hills with nonunique stationary points are treated. A framework has been provided, with which convergence of stochastic versions of conjugate gradient, partan, etc., can be discussed and proved.