Compression-based methods for nonparametric density estimation, on-line prediction, regression and classification for time series

We address the problem of nonparametric estimation of characteristics for stationary and ergodic time series. We consider finite-alphabet time series and the real-valued ones and the following problems: estimation of the (limiting) probability P(u_{0 hellip} u_s) for every s and each sequence u₀ hellip u_s of letters from the process alphabet (or estimation of the density p(x₀, _hellip , x_s) for real-valued time series), so-called on-line prediction, where the conditional probability P(x_t+1/x₁x_{2 hellip} x_t) (or the conditional density p(x_t+1/x₁x_{2 hellip} x_t)) should be estimated (in the case where x₁x₂ hellip x_t is known), regression and classification (or so-called problems with side information). We show that any universal code (or a universal data compressor) can be used as a basis for constructing asymptotically optimal methods for the above problems.