A probabilistic framework for combining multiple classifiers at abstract level

Most previous studies assumed that classifiers behave independently. Such an assumption degrades and biases classification performance in the case of adding highly dependent classifiers. In order to overcome such a weakness, one should consider combining multiple classifiers in a probabilistic framework using a Bayesian formalism without the assumption. The probabilistic combination of K classifiers needs a (K+1)st-order probability distribution. However, it as well known that the distribution becomes unmanageable when storing and estimating, even for small K. Chow and Liu (1968) as well as Lewis (1959) proposed a product approximation of a high order distribution with a set of only first-order tree dependencies or second-order distributions. However, if a classifier follows more than one classifier, such a first-order dependency will not be suitable to approximate the high order distribution properly. A probabilistic framework is proposed to identify an optimal product set of kth-order dependencies for an approximation of the (K+1)st-order probability distribution where 1⩽k⩽K, and to combine K decisions by the identified product set using a Bayesian formalism. This framework was experimented and evaluated with a standardized CENPARMI database and showed superior performance when compared to other combination methods