Almost sure convergence of two time-scale stochastic approximation algorithms

The almost sure convergence of two time-scale stochastic approximation algorithms is analyzed under general noise and stability conditions. In the context of the Lyapunov stability, the adopted stability conditions are probably the weakest possible still allowing the almost sure convergence to be shown, while the corresponding noise conditions are the most general ones under which the almost sure convergence analysis can be carried out. The analysis covers the algorithms with additive noise, as well as those with non-additive noise. The algorithms with additive noise are analyzed for the case where the noise is state-dependent. The analysis of the algorithms with non-additive state-dependent noise is carried out for the case where the noise is a Markov chain controlled by the algorithm states, while the algorithms with non-additive exogenous noise are analyzed for the case where the noise is correlated and satisfies strong mixing conditions. The obtained results cover a fairly broad class of highly non-linear two time-scale stochastic approximation algorithms.