Universal schemes for learning the best nonlinear predictor given the infinite past and side information

Let {X_t} be a real-valued time series. The best nonlinear predictor of X₀ given the infinite past X_-∞^-1 in the least squares sense, is equal to the conditional mean E{X₀|X_-∞^-1}. Previously, it has been shown that certain predictors based on growing segments of past observations converge to the best predictor given the infinite past whenever {X_t} is a stationary process with values in a bounded interval. The present paper deals with universal prediction schemes for stationary processes with finite mean. We also discuss universal schemes for learning the conditional mean E{X₀|X _-∞^-1Y_-∞^-1Y₀ } from past observations of a stationary pair process {(X_t , Y_t)}, and schemes for learning the repression function m(y)=E{X|Y=y} from independent samples of (X, Y)