Feature Selection for Classification Using Decision Tree

In most pattern recognition (PR) system, selecting the best feature vectors is an important task. Feature vectors serve as a reduced representation of the original data/signal/input that helps avoid the curse of dimensionality in a PR task. In this work, we consider further effort in selecting the best feature vectors for the PR task. A framework to determine the best eigenvectors of two main human postures based on the rules of thumb of principal component analysis (PCA) has been developed, which includes the KG-rule, cumulative variance and the scree test. Accordingly, two rules of thumb suggest in retaining only the first six eigenvectors or also known as ´eigenposture´, as inputs for classification. Using decision tree (DT) as our classifier, two distinct processes need to be implemented namely, building the tree and then performing classification using the top-down approach in DT construction. At each node, decision to what is likely to be the best split is determined using the predicted value. Each branch in the tree is labeled with its decision rule whilst each terminal node is labeled with the predicted value of that node. Cross validation ensures selection of an optimum tree size and helps avoid the problem of over fitting. Consequently, the framework has enabled us to select the best and most optimized eigenpostures for classification purpose.