Online optimization in parametric dynamic environments

This paper describes a novel approach to online convex programming in dynamic settings. Many existing online learning methods are characterized via regret bounds that quantify the gap between the performance of the online algorithm relative to a comparator. In previous work, this comparator was either considered static over time, or admitted sublinear tracking regret bounds only when the comparator was piecewise constant or slowly varying. In contrast, the proposed method determines the best dynamical model from a parametric family to incorporate into the prediction process, and admits regret bounds which scale with the deviation of a comparator from that dynamical model. In other words, this approach can yield low regret relative to highly variable, dynamic comparators. This result is proved for loss functions corresponding to the negative log likelihood associated with an exponential family probability distribution, and several properties of the exponential family are exploited to ensure low regret.