Dynamic Programming as Applied to Feature Subset Selection in a Pattern Recognition System

This paper proposes dynamic programming search procedures to expedite the feature subset selection processes in a pattern recognition system. It is shown that in general the proposed procedures require much fewer subsets to be evaluated than the exhaustive search procedure. For example, a problem of selecting the best subset of 4 features from a set of 24 features requires an evaluation of (24/4) = 10626 subsets by using the exhaustive search procedure; on the other hand, it requires only 175 and 136 subsets to be considered by employing the proposed Procedures I and II, respectively, to solve the same problem. While the number of subsets to be evaluated for the dynamic programming search procedures is slightly greater than that for the without-replacement search procedure, the best feature subset selected by the proposed methods may, however, not necessarily contain all of the best single features selected in the previous stages.