Convergence and loss bounds for Bayesian sequence prediction

The probability of observing x/sub t/ at time t, given past observations x/sub 1/...x/sub t-1/ can be computed if the true generating distribution (mu)of the sequences x/sub 1/x/sub 2/x/sub 3/... is known. If (mu)is unknown, but known to belong to a class M one can base oneʹs prediction on the Bayes mix (xi) defined as a weighted sum of distributions (nu) (is a member of) M. Various convergence results of the mixture posterior (xi)/sub t/ to the true posterior (mu)/sub t/ are presented. In particular, a new (elementary) derivation of the convergence (xi)/sub t//(mu)/sub t/ - 1 is provided, which additionally gives the rate of convergence. A general sequence predictor is allowed to choose an action y/sub t/ based on x/sub 1/...x/sub t-1/ and receives loss L/sub x(t)y(t)/ if x/sub t/ is the next symbol of the sequence. No assumptions are made on the structure of L (apart from being bounded) and M. The Bayes-optimal prediction scheme (lambda)/sub (xi)/ based on mixture (xi) and the Bayes-optimal informed prediction scheme (lambda)/sub (mu)/ are defined and the total loss L/sub (xi)/ of (lambda)/sub (xi)/ is bounded in terms of the total loss L/sub (mu)/ of (lambda)/sub (mu)/. It is shown that L/sub (xi)/ is bounded for bounded L/sub (mu)/ and L/sub (xi)/L/sub (mu)/ - 1 for L/sub (mu)/ - (infinity). Convergence of the instantaneous losses is also proven.