Stochastic approximation with discontinuous dynamics and state dependent noise: W. P. 1 convergence

Stochastic approximations of the form Xn+1 = Xn + anh(Xn, ??n) are treated where h(?? , ??) might not be continuous and the noise sequence {??n} might depend on {Xn}. An ´averaging´ and an ´ordinary differential equation´ method are combined to get w.p.1 convergence for both the above algorithm and for the case where the iterates are projected back onto a bounded set G if they ever leave it. Two examples are developed, the first being an automata problem where the dynamics are not smooth and the noise is state dependent, and the second a Robbins-Monro process with observation averaging (which causes the noise to be state dependent). Each example is typical of a larger class.