Transductive learning algorithms for nonstationary environments

Many traditional supervised machine learning approaches, either on-line or batch based, assume that data are sampled from a fixed yet unknown source distribution. Most incremental learning algorithms also make the same assumption, even though new data are presented over periods of time. Yet, many real-world problems are characterized by data whose distribution change over time, which implies that a classifier may no longer be reliable on future data, a problem commonly referred to as concept drift or learning in nonstationary environments. The issue is further complicated when the problem requires prediction from data obtained at a future time step, for which the labels are not yet available. In this work, we present a transductive learning methodology that uses probabilistic models to aid in computing ensemble classifier voting weights. Assuming the drift is limited in nature, the proposed approach exploits a probabilistic estimate to determine the class responsibility of components in a Gaussian mixture model (GMM), generated from labeled and unlabeled data. A general error bound is provided based on the ensemble decision, the probabilistic estimate of the GMM, and the true labeling function, which, unfortunately is never actually known.