Near-Optimal Approximation Rates for Distribution Free Learning with Exponentially, Mixing Observations

This paper derives the rate of convergence for the distribution free learning problem when the observation process is an exponentially strongly mixing (α-mixing with an exponential rate) Markov chain. If {z_k}_K=1^∞ = {(x_k, y_k)}_k=1^∞ ⊂ x × Y ≡ Z is an exponentially strongly mixing Markov chain with stationary measure ρ, it is shown that the empirical estimate f_z that minimizes the discrete quadratic risk satisfies the bound E_z∈Z^m (∥ f_ρ - f_z ∥_L2(ρx)) ≤ C (lna/a)^r/(2r+1) where E_z∈Z^m (·) is the expectation over the first m-steps of the chain, f_ρ is the regressor function in L²_(ρX) associated with ρ, r is related to the abstract smoothness of the regressor, ρ_X is the marginal measure associated with ρ, and a is the rate of concentration of the Markov chain.