On distributed stochastic optimization of regenerative systems using IPA

The use of a distributed asynchronous algorithm utilizing infinitesimal perturbation analysis (IPA) gradient estimators for online multivariable optimization of systems that have regenerative properties is described. Each control variable has a processor that performs updates of the parameter according to a stochastic gradient algorithm driven by the IPA gradient estimates. The update times of the processors are not synchronized. The processors also communicate results of computations with each other, and this communication involves delays. Conditions are given under which the algorithm converges with probability one to the optimal parameter value. In their proof of convergence, the authors analyze a particular subsequence of the sequence of control parameters, and show that this sequence behaves like a sequence generated by a centralized synchronous gradient algorithm that updates before the start of each cycle of the system, and with gradient estimates that are asymptotically unbiased