Dimension reduction using least squares regression in multi-labeled text categorization

Dimension reduction is a preprocessing step by which small number of optimal features are extracted. Among several statistical dimension reduction methods, Linear discriminant analysis (LDA) performs dimension reduction to maximize class separability in the reduced dimensional space. However, in multi-labeled problems, data samples belonging to multiple classes cause contradiction between the maximization of the distances between classes and the minimization of the scatter within classes, since they are placed in the overlapping area of multiple classes. In this paper, we show that in multi-labeled text categorization, the outputs from multiple linear methods can be used to compose new features for low dimensional representation. Especially, we apply least squares regression and a linear support vector machine (SVM) for multiple binary-class problems constructed from a multi-labeled problem and obtain optimal features in a low dimensional space which are fed into another classification algorithm. Extensive experimental results in text categorization are presented comparing with other dimension reduction methods and multi-label classification algorithms.