Class-specific feature sets in classification

The classical Bayesian approach to classification requires knowledge of the probability density function (PDF) of the data or sufficient statistic under all class hypotheses. Since it is difficult or impossible to obtain a single low-dimensional sufficient statistic, it is often necessary to utilize a sub-optimal yet still relatively high-dimensional feature set. The performance of such an approach is severely limited by the ability to estimate the PDF on a high-dimensional space from training data. A new theorem shows that by introducing a special “noise-only” signal class, it is possible to re-formulate the classical approach based upon M sufficient statistics, one corresponding to each signal class. Also, the optimal classifier requires knowledge of only the PDF´s sufficient statistics under the corresponding signal class and under noise-only condition. We present simulation results of a 9-class synthetic problem showing dramatic improvements over the traditional high-dimensional approach