Clustering Inside Classes Improves Performance of Linear Classifiers

This work systematically examines a Clustering Inside Classes (CIC) approach to classification. In CIC, each class is partitioned into subclasses based on cluster analysis. We find that CIC, by extracting local structure and producing compact subclasses, can improve performance of linear classifiers such as SVM and logistic regression. It is compared against a global classifier on four benchmark datasets. We empirically analyze effects of the training set size and the number of clusters per class on the results of the CIC approach. We also examine use of an automated method for selecting the number of clusters for each class.