DocumentCode :
1798221
Title :
Learning optimization for decision tree classification of non-categorical data with information gain impurity criterion
Author :
Sofeikov, K.I. ; Tyukin, I.Yu. ; Gorban, A.N. ; Mirkes, E.M. ; Prokhorov, Danil V. ; Romanenko, I.V.
Author_Institution :
Dept. of Math., Univ. of Leicester, Leicester, UK
fYear :
2014
fDate :
6-11 July 2014
Firstpage :
3548
Lastpage :
3555
Abstract :
We consider the problem of construction of decision trees in cases when data is non-categorical and is inherently high-dimensional. Using conventional tree growing algorithms that either rely on univariate splits or employ direct search methods for determining multivariate splitting conditions is computationally prohibitive. On the other hand application of standard optimization methods for finding locally optimal splitting conditions is obstructed by abundance of local minima and discontinuities of classical goodness functions such as e.g. information gain or Gini impurity. In order to avoid this limitation a method to generate smoothed replacement for measuring impurity of splits is proposed. This enables to use vast number of efficient optimization techniques for finding locally optimal splits and, at the same time, decreases the number of local minima. The approach is illustrated with examples.
Keywords :
decision trees; learning (artificial intelligence); optimisation; pattern classification; Gini impurity; classical goodness functions; conventional tree growing algorithms; direct search methods; high-dimensional data; information gain impurity criterion; learning optimization; multivariate splitting conditions; noncategorical data decision tree classification; split impurity; univariate splits; Decision trees; Electronic mail; Entropy; Hafnium; Impurities; Optimization; Standards;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Neural Networks (IJCNN), 2014 International Joint Conference on
Conference_Location :
Beijing
Print_ISBN :
978-1-4799-6627-1
Type :
conf
DOI :
10.1109/IJCNN.2014.6889842
Filename :
6889842
Link To Document :
بازگشت