Induction of Multi-stage decision tree

Motivated by improving the existing decision tree performance of dealing with multi-class problems, this paper proposes a new algorithm named multi-stage decision tree (MDT). The MDT algorithm is based on the relationship between the margin of SVM hyper-planes and their generalization capability and tries to find the large margin among the clusters. First the MDT algorithm converts the multi-class problem into two-class problem by large margin learning of SVM hyper-planes, and then for each two-class problem, it uses traditional decision tree induction algorithm to generate a decision tree which splits a dataset into two subsets for the further induction. Recursively, the multi-stage decision tree is obtained finally. Numerical simulations show the effectiveness and high accuracy of MDT algorithm. Initial experiments show that the MDT algorithm has the potential application to text classification with many classes.