Decompose Learning: Combine Feature Extraction and Classification

In real world classification tasks, the original instances are represented by raw features. Usually domain related algorithms are needed to extract discriminative features. But the algorithms selection and additional parameters tuning are difficult for people with little domain knowledge and experience. In this paper, a new machine learning framework called "decompose learning" is proposed for classification tasks with raw features. The raw input space is decomposed into several subspaces and an independent base classifier is learned in each subspace. The predictive result of each base classifier can be regarded as a high-level feature of original task. The final classifier is learned on these high-level features. We model the decomposition as a "subspace clustering" problem and utilize target unrelated unlabeled data to extract target related subspaces. Besides that, we use a maximum margin base classifier selection strategy to do the capacity control. Empirical tests on MNIST and Caltech101 datasets show that decompose learning can improve predictive accuracy considerably without highly specialized domain related feature extraction algorithms.