On prediction of individual sequences relative to a set of experts

We investigate sequential randomized prediction of an arbitrary binary sequence. The goal of the predictor is to minimize Hamming loss relative to the loss of the best “expert” in a fixed set of experts. We point out a close connection between the prediction problem and empirical process theory. We show upper and lower bounds on the minimax relative loss in terms of the geometry of the class of experts. As a main example, we determine the exact order of magnitude of the minimax relative loss for the class of Markov experts. Furthermore, in the special case of static experts, we completely characterize the minimax relative loss in terms of the maximal deviation of an associated Rademacher process