On Probability Estimation by Exponential Smoothing

Probability estimation is an elementary building block of every statistical data compression algorithm. In practice probability estimation should be adaptive, recent observations should receive a higher weight than older observations. We present a probability estimation method based on exponential smoothing that satisfies this requirement. Our main contribution is a theoretical analysis for various smoothing rate sequences: We show that the redundancy w.r.t. A piecewise stationary model with s segments is O (s n0.5) for any bit sequence of length n, an improvement over previous approaches with a similar complexity.