Just-in-time ensemble of classifiers

Handling dynamic environments and building up algorithms operating at low supervised-sample rates are two main challenges for classification systems designed to operate in real-life scenarios. Here, changes in the probability density function of classes characterizing the data-generating process (also called concept drift) should be detected as soon as possible to prevent the classifier from becoming obsolete. Moreover, when the rate of supervised samples during the operational life is low (as in those situations where the sample inspection is costly or destructive) both detecting the change and re-training the classifier become even more critical aspects. We present an adaptive classifier that exploits both supervised and unsupervised data to monitor the process stationarity. The classifier follows the just-in-time (JIT) approach and relies on two different change-detection tests (CDTs) to reveal changes in the environment and reconfigure the classifier accordingly. The proposed solution assesses the stationary in both the joint probability density function (CDT at the classification error) and the distribution of the inputs (CDT on unlabeled data). In addition, we integrate in the JIT adaptive classifier a procedure able to handle recurrent concepts within an ensemble of classifiers framework. Experiments show that monitoring unsupervised samples and handling recurrent concepts is essential for classifying in non-stationary environments when few supervised samples are available.