Improving classification decisions by multiple knowledge

An important issue in data mining is how to make use of multiple discovered knowledge to improve future decisions. In this paper, we propose a new approach to combining multiple sets of rules for text categorization using Dempster´s rule of combination. We develop a boosting-like technique for generating multiple sets of rules based on rough set theory and model classification decisions from multiple sets of rules as pieces of evidence which can be combined by Dempster´s rule of combination. We apply these methods to 10 out of the 20-newsgroups - a benchmark data collection, individually and in combination. Our experimental results show that the performance of the best combination of the multiple sets of rules on the 10 groups of the benchmark data is statistically significantly better than that of the best single set of rules. The comparative analysis between the Dempster-Shafer and the majority voting methods along with an overfitting study confirm the advantage and the robustness of our approach