Time-average stochastic optimization with non-convex decision set and its convergence

This paper considers time-average stochastic optimization, where a time average decision vector, an average of decision vectors chosen in every time step from a time-varying (possibly non-convex) set, minimizes a convex objective function and satisfies convex constraints. This formulation has applications in networking and operations research. In general, time-average stochastic optimization can be solved by a Lyapunov optimization technique. This paper shows that the technique exhibits a transient phase and a steady state phase. When the problem has a unique vector of Lagrange multipliers, the convergence time can be improved. By starting the time average in the steady state, the convergence times become O(1/ε) under a locally-polyhedral assumption and O(1/ε^1.5) under a locally-non-polyhedral assumption, where e denotes the proximity to the optimal objective cost.