Growing a fuzzy decision forest

The creation of multiple decision trees is a relatively new concept, which aims to improve the predictive power of a single decision tree. The approach is based on the induction of more than one C4.5-type decision tree from the same training sample where each decision tree represents a different view of the same domain resulting in a network of decision tree models. The utilization of multiple decision trees has been shown to lead to an improved performance by combining multiple perspectives of the same domain thus increasing the information content whereas, in comparison, a single decision tree can only represent one restricted view of the domain. One predominant weakness in creating a single tree is the generation of sharp decision boundaries at every node within the tree, which results in small changes in attribute values giving radically different classifications. This problem becomes more apparent with the generation of multiple trees. This paper presents a novel approach of overcoming this weakness through the use of fuzzy decision forests. The approach is based upon the induction of multiple fuzzy decision trees from one training sample, where each tree represents a different view of the data domain. A genetic algorithm (GA) is used to select a series of high performance membership functions, which are then applied to branches within all decision trees in the forest. The GA will in addition optimise a pre-selected fuzzy inference technique, which will assign a degree of strength to the conjunction and disjunction of membership grades within the tree. Considerable improvements in classification accuracy over original single C4.5 (crisp) trees were obtained using two real world data sets.